In the vein of my last question, I'm now at a roadblock on question 3 of this sheet:
http://www.trin.cam.ac.uk/dpk10/IA/exsheet3.pdf
(note: it's not my intention to ask every question I get stuck on here, merely the ones which have interesting general results; it just so happens that the two I've struggled with so far fit this criteria)
It goes as follows:
Let $N$ be a non-negative integer-valued random variable with mean $\mu_1$ and variance $\sigma_1^2$, and let $X_1, X_2, …$ be random variables, each with mean $\mu_2$ and variance $\sigma_2^2$; furthermore, assume that $N, X_1, X_2, . . .$ are independent. Without using generating functions, calculate the mean and variance of the random variable $S_N = X_1 + … + X_N$ (when $N=0$ interpret $S_N$ as $0$).
I have the answer to calculating the mean, which I'll write up as an answer for future reference. Here's where I've got to with the variance bit:
$\mathbb{E}(S_N^2) = 0.P(N=0) + \sum_{r=1}^\infty\mathbb{E}((\sum_{i=1}^rX_i)^2).P(N=r)$
For $r=1$, we have the inner expectation as: $\mathbb{E}(X_1^2)$ which is equal to $\sigma_2^2+\mu_2^2$.
For $r > 1$, we have the inner expectation equal to: $r(\sigma_2^2 + 2\mu_2^2)$
So $\mathbb{E}(S_N^2) = (\sigma_2^2+\mu_2^2)P(N=1) + (\sigma_2^2 + 2\mu_2^2)\sum_{r=2}^\infty rP(N=r)$
which is almost in the form where I can use the same trick as in calculating the mean, but not quite.
Any help is much appreciated.
Best Answer
The easy way is to use the law of total variance:
$$\text{Var}(S) = E_N\left[\text{Var}(S|N)\right] + \text{Var}_N\left[E(S|N)\right] =\text{E}_N\left[N\cdot \text{Var}(X)\right] + \text{Var}_N\left[N\cdot\text{E}(X)\right]$$
Can you do it from there? It's pretty much just substitution (well, that and really basic properties of expectation and variance).
(The first part is even more straightforward using the law of total expectation.)
--
As Spy_Lord notes, the answer is $\text{E}(N)\cdot \text{Var}(X) + \text{Var}(N)\cdot\text{E}(X)^2$
Alternative approach is to evaluate $E(S_N^2)$. Following the approach you seem to be aiming at:
\begin{eqnarray} E(S_N^2) &=& \sum_r E(S_N^2|N=r) p_r\\ &=& \sum_r (r\sigma_2^2+r^2 \mu_2^2) p_r\\ &=& \sigma_2^2\sum_r rp_r+\mu_2^2\sum_rr^2 p_r \\ &=& \sigma_2^2 \text{E}N+\mu_2^2\text{E}(N^2) \end{eqnarray}
and I assume you can do it from there.
However, to be honest, I think this way is easier (it's actually the same approach, you just don't need to sum over all the mutually exclusive events that way). The law of total expectation says $\text{E}(X) = \text{E}_Y[\text{E}_{X|Y}(X|Y)]$, so
\begin{eqnarray} \text{E}(S^2_N) &=& \text{E}_N[\text{E}(S^2_N|N)]\\ &=& \text{E}_N[N\sigma_2^2+N^2\mu_2^2]\\ &=& \sigma_2^2\text{E}(N)+\mu_2^2\text{E}(N^2) \end{eqnarray}