Suppose that $A_1$ and $A_2$ are iid $\text{Exp}(2)$ and let $X_1 = A_1$, $X_2 = A_1 + A_2$. Find $\operatorname{Cov}(X_1, X_2)$.
To find covariance, I first found the variance for $X_1$ and $X_2$. I got:
$\operatorname{Var}(X_1) = 1/4$
$\operatorname{Var}(X_2) = 1/2$
I know that the covariance formula is $\operatorname{Cov}(X_1,X_2) = E[(X_1-E[X_1])(X_2-E[X_2])]$, but I am having trouble computing this. Is this just the standard deviation of $X_1$ $\cdot$ standard deviation of $X_2$?
Best Answer
Use instead the formula $$\operatorname{Cov}[X_1,X_2] = \operatorname{E}[X_1 X_2] - \operatorname{E}[X_1]\operatorname{E}[X_2].$$ Then we have by linearity of expectation $$\operatorname{E}[X_1 X_2] = \operatorname{E}[A_1 (A_1 + A_2)] = \operatorname{E}[A_1^2 + A_1 A_2] = \operatorname{E}[A_1^2] + \operatorname{E}[A_1 A_2].$$ Then since $$\operatorname{Var}[A_1] = \operatorname{E}[A_1^2] - \operatorname{E}[A_1]^2,$$ and the independence of $A_1$ and $A_2$ implies $$\operatorname{E}[A_1 A_2] = \operatorname{E}[A_1]\operatorname{E}[A_2],$$ it follows that $$\operatorname{E}[X_1 X_2] = \operatorname{Var}[A_1] + \operatorname{E}[A_1]^2 + \operatorname{E}[A_1]\operatorname{E}[A_2].$$ Then $$\begin{align} \operatorname{Cov}[X_1, X_2] &= \operatorname{Var}[A_1] + \operatorname{E}[A_1]^2 + \operatorname{E}[A_1]\operatorname{E}[A_2] - \operatorname{E}[A_1]\operatorname{E}[A_1 + A_2] \\ &= \operatorname{Var}[A_1] + \operatorname{E}[A_1]^2 + \operatorname{E}[A_1]\operatorname{E}[A_2] - \operatorname{E}[A_1](\operatorname{E}[A_1] + \operatorname{E}[A_2]) \\ &= \operatorname{Var}[A_1] + \operatorname{E}[A_1]^2 + \operatorname{E}[A_1]\operatorname{E}[A_2] - \operatorname{E}[A_1]^2 - \operatorname{E}[A_1]\operatorname{E}[A_2] \\ &= \operatorname{Var}[A_1]. \end{align}$$ This of course should not be surprising: because $A_1$ and $A_2$ are independent, the extent to which $X_1 = A_1$ and $X_2 = A_1 + A_2$ vary with each other is simply the extent to which $A_1$ varies, since the addition of $A_2$ does not contribute any additional covariance. This also happens to be a special case of the more general identity $$\operatorname{Cov}[aX+bY, cW+dV] = ac \operatorname{Cov}[X,W] + ad \operatorname{Cov}[X,V] + bc \operatorname{Cov}[Y,W] + bd \operatorname{Cov}[Y,V],$$ with $a = c = d = 1$, $b = 0$, $X = W = A_1$, $V = A_2$. Then we see that since the only pair of variables with nonzero covariance is the first one, the formula reduces to $\operatorname{Cov}[A_1, A_1] = \operatorname{Var}[A_1]$.