I often see the term white noise appearing when reading about different statistical models. I must however admit, that I am not completely sure what this means. It is usually abbreviated as $WN(0,σ^2)$. Does that mean it's normally distributed or could it follow any distribution?
Solved – White Noise in Statistics
normal distributionwhite noise
Related Solutions
Regardless of whether $X$ and $Y$ are normal or not, it is true (whenever the various expectations exist) that \begin{align} \mu_{X+Y} &= \mu_X + \mu_Y\\ \sigma_{X+Y}^2 &= \sigma_{X}^2 + \sigma_{Y}^2 + 2\operatorname{cov}(X,Y) \end{align} where $\operatorname{cov}(X,Y)=0$ whenever $X$ and $Y$ are independent or uncorrelated. The only issue is whether $X+Y$ is normal or not and the answer to this is that $X+Y$ is normal when $X$ and $Y$ are jointly normal (including, as a special case, when $X$ and $Y$ are independent random variables). To forestall the inevitable follow-up question,
No, $X$ and $Y$ being uncorrelated normal random variables does not suffice to assert normality of $X+Y$. If $X$ and $Y$ are jointly normal, then they also are marginally normal. If they are jointly normal as well as uncorrelated, then they are marginally normal (as stated in the previous sentence) and they are independent as well. But, regardless of whether they are independent or dependent, correlated or uncorrelated, the sum of jointly normal random variables has a normal distribution with mean and variance as given above.
In a comment following this answer, ssdecontrol raised another question: is joint normality just a sufficient condition to assert normality of $X+Y$, or is it necessary as well?
Is it possible to find marginally normal $X$ and $Y$ that are not jointly normal but their sum $X+Y$ is normal? This question was asked in the comments below by Moderator Glen_b. This is indeed possible, and I have given an example in an answer to this question.
Is it possible to find $X$ and $Y$ such that they are not jointly normal but their sum $X+Y$ is normal? Here, we do not insist on $X$ and $Y$ being marginally normal. The answer is Yes, and an example is given by kjetil b halvorsen. Another, perhaps simpler, answer is as follows. Consider $U$ and $V$ be independent standard normal random variables and $W$ a discrete random variable taking on each of the values $+1$ and $-1$ with probability $\frac 12$. Then, $X = U+W$ and $Y=V-W$ are not marginally normal (they have identical mixture Gaussian density $\frac{\phi(t+1)+\phi(t-1)}{2}$), and so are not jointly normal either. But their sum $X+Y = U+V$ is a $N(0,2)$ random variable.
Question1: If the power spectrum is not flat, then does that mean the colored noises are correlated?
One way to construct the power spectral density is to take the Fourier transform of the autocovariance function. This result is called the Wiener–Khinchin theorem. According to the theorem the autocovariance function is
$$ R_{xx}(t) = \int_{-\infty}^{\infty} = S(f)e^{2\pi itf} df, $$
where $S(f)$ is the power spectral density. The expression $e^{2\pi itf}$ describes rotations around a circle at frequency $f$. Integrating the area this rotation traces out turns out to be 0, which implies that the autocovariance function is flat at 0 only if $S(f)$ is flat. In order to compute the autocorrelation function you have to normalise by the variance of the noise, i.e. $R_{xx}(t)/R_{xx}(0)$.
To examine autocorrelation functions for different colours of noise I simulate 1000 timeseries of length 1000 each:
As can be seen, all colours of noise apart from white are autocorrelated for $t=1$. However, the correlation for blue and violet noise is negative and for larger $t$'s violet noise is not correlated. This is because violet noise is constructed by differentiating white noise, which means that every sample is the difference between two Gaussian random numbers. If we call these Gaussian numbers $w_1,w_2,\dots$ and the samples of violet noise $v_1,v_2,\dots$ we can write
$$v_1 = w_1 - w_2, v_2 = w_2 - w_3, v_3 = w_3 - w_4,\dots$$.
Note that the Gaussian random number $w_2$ is part of $v_1$ and $v_2$ with opposite signs (which explains the negative correlation) but not of $v_3$ (which explains that it is not correlated for $t>1$).
To illustrate this further I have also plotted the partial autocorrelation, which is the autocorrelation when controlling for all smaller $t$'s. Here violet noise is partially autocorrelated throughout. The reason is that the estimate of $v_3$ from $v_2$ can be improved by also knowing $v_1$. This is because $v_1$ contains in formation about $w_2$, which can be used to more accurately estimate the $w_3$ part of $v_2$, which in turn is needed to estimate $v_3$. (Note that the partial autocorrelation for violet noise would be flat if $t=1$ were not included.)
Another interesting observation is that the partial autocorrelation for brown noise beyond $t=1$ is flat at 0. This is because brown noise is obtained by integrating white noise, which makes every sample the cumulative sum of all previous Gaussian random numbers. The sample at $t=1$ is not crucial for estimating the current sample (thus the high autocorrelation for $t>1$) but if the sample at $t=1$ is known, then any sample $t>1$ does not add any new information (thus the low partial autocorrelation for $t>1$).
Pink and blue noise can be considered cases between these two extremes.
As a final aside, the autocorrelation for an infinite timeseries of brown noise is flat at 1 because the autocovariance $R_{xx}(t)$ and the variance $R_{xx}(0)$ are approaching infinity at the same rate.
Question2: The plot below is the time series which is the output of the sum of white Gaussian noise and pink noise. I don't know if pink noise is correlated or not. In general, does the addition of an uncorrelated r.v with a correlated r.v give an output that is correlated or uncorrelated? i.e., z=x+y is z correlated/uncorrelated?
As you can see from the above plot pink noise is autocorrelated. You can obtain this result yourself from your simulated data by shifting its values by different lags $t$ and computing the correlation for each lag. Generally, if you add a correlated timeseries to an uncorrelated one the resulting timeseries will be correlated but less so.
Best Answer
TL;DR
The answer is NO, it doesn't have to be normal; YES, it can be other distributions.
Colors of the noise
Let's talk about colors of the noise.
If you use noise cancelling head phones, you know that #1 is impossible to cancel. It'll pierce through any head phone with ease. #2 will be cancelled very well.
As to #3, why would you cancel it?
Origin of a term "color"
What's the distinction between these three noises? It comes from spectral analysis. As you know from high school years you can send the white light through a prism and it will split the light into all different colors. That's what we call white: all colors in approximately the same proportion. No color dominates.
The color is the light of a certain frequency, or you could say electromagnetic waves of certain wavelength like shown below. The red color has low frequency relative to the blue, equivalently the red color has longer wavelength of almost 800nm compared to the blue wavelength of 450nm.
Spectral Analysis
If you take noise, whether acoustic, radio or other, and send it through the spectral analysis tool such as FFT, you get it spectral decomposition. You'll see how much of each frequency is in the noise, like shown in the next picture from Wikipedia. It's clear that this is not white noise: it has clear peaks at 50Hz, 40Hz etc.
If a narrow frequency band sticks out, then it's called colored, as in not white. So, white noise is just like white light, it has a wide range of frequencies in approximately same proportion like shown in the next figure from this site. The top chart shows the recording of the amplitude, and the bottom shows the spectral decomposition. No frequency sticks out. So the noise is white.
Perfect sine
Now, why does the sequence of independent identically distributed random numbers(iid) generates the white noise? Let's think of what makes a signal colored. It's the waves of certain frequency sticking out from others. They dominate the spectrum. Consider a perfect sign wave: $\sin(2\pi t)$. Let's see what is the covariance between any two points $\phi=1/2$ seconds apart: $$E[\sin(2\pi t) \times \sin(2\pi (t+1/2)]=-E[\sin^2 (2\pi t)]=-\frac 1 2$$
So, in the presence of the sine wave we'll get autocorrelation in the time series: all oservations half a second apart will be perfectly negatively correlated! Now, saying that our data is i.i.d. implies that there is no any autocorrelation whatsoever. This means that there are no waves in the signal. The spectrum of the noise is flat.
Imperfect Example
Here's an example I created on my computer. I first recorded my tuning fork, then I recorded the noise from computer's fans. Then I ran the following MATLAB code to analyze the spectra:
Here's the signal and the spectrum of the tuning fork. As expected it has a peak at around 440Hz. The tuning fork must produce a nearly ideal sine wave signal, like in my theoretical example earlier.
Next I did the same to the noise. As expected no frequency is sticking out. Obviously this is not the white noise, but it gets quite close to it. I think that there must be very high pitched frequency, it bother me a little bit. I need to change the fan soon. However, I don't see it in the spectrum. Maybe because my microphone is beyond crappy, or the sampling frequency is not high enough.
Distribution doesn't matter
The important part is that in the random sequence the numbers are not autocorrelated (or even stronger, independent). The exact distribution is not important. It could be Gaussian or gamma, but as long as the numbers do not correlate in the sequence the noise will be white.