Solved – Why does the variance decrease when averaging zero-mean gaussians

meannormal distribution

I have the following in my textbook:

$$
r_k \thicksim N(0,\sigma_r^2) \\ \Rightarrow
\sum\limits_{k=1}^K r_k(t) \thicksim N(0,K\sigma_r^2) \\ \Rightarrow
\frac1{K} \sum\limits_{k=1}^K r_k(t) \thicksim N(0,\frac{K}{K^2}\sigma_r^2) = N(0,\frac1{K}\sigma_r^2)
$$

The context is averaging noise away. I have a feeling that this is correct but I can't see it mathematically:
Question 1:
How do I get from here:
$$\sum\limits_{k=1}^K N(0,\sigma_r^2) = \sum\limits_{k=1}^K \frac1{\sqrt{2\pi \sigma_r^2}}e^{-\frac{(x-0)^2}{2\sigma_r^2}}$$
to here:
$$N(0,K\sigma_r^2) = \frac1{\sqrt{2\pi K\sigma_r^2}}e^{-\frac{(x-0)^2}{2K\sigma_r^2}}$$

solved by aussetg, many thanks. I was interpreting $\sim$ as a fancy $=$. I learned that this is wrong: The sum of 2 dices is not twice the sum of one dice, compare probability of getting a seven.
Knowing that given
$X \thicksim N(\mu_X, \sigma_X^2),
Y \thicksim N(\mu_Y, \sigma_Y^2)$
$Z(=X+Y) \thicksim N(\mu_X + \mu_Y, \sigma_X^2+\sigma_Y^2)$
It becomes clear that $\sum\limits_{k=1}^K r_k(t) \thicksim N(0,K\sigma_r^2)$

Question 2
how to get from here:
$$\frac1{K} N(0,K\sigma_r^2) =\frac1{K} \frac1{\sqrt{2\pi K\sigma_r^2}}e^{-\frac{(x-0)^2}{2K\sigma_r^2}}$$
to here:
$$N(0,\frac1{K}\sigma_r^2) = \frac1{\sqrt{2\pi \frac1{K}\sigma_r^2}}e^{-\frac{(x-0)^2}{2\frac1{K}\sigma_r^2}}$$

This is really easy once you realise $\sim$ != $=$.
If $var(X) = \sum\limits_i (\bar{x} – x_i)^2$ then $var(aX) = \sum\limits_i (a\bar{x} – ax_i)^2 = a^2\sum\limits_i (\bar{x} – x_i)^2 = a^2var(x)$
Therefore if $r'\sim N(0,K\sigma_r^2) \Rightarrow \frac1{K}r'\sim N(0,(\frac1{K})^2K\sigma_r^2)$

Best Answer

Don't sum densities, it doesn't mean anything. In particular, the density of a sum of random variables is not the sum of the densities. You may want to consider the case of rolling one vs. two dice to gain intuition here. Here's a reference.

What you need to do is prove by induction that the sum of independant gaussians is a gaussian.

The reduction of variance part in itself is trivially obtained by the definition

$$ \mathbb{V}(X) = \mathbb{E} [ (X-\mathbb{E}(X))^2 ] $$

by observing it's a quadratic form. ( the covariance is actually a bilinear symmetric function ). Just plug your sum and use the independance of the variables, you will find that the sum contribute only to a factor $k$ while the leading $1/k$ contribute to $1/k^2$.

Related Question