Solved – Covariance of conditionally independent random variables

conditional-expectationcovariance

I want to find the covariance between two random variables $X$ and $Y$, which are independent given another random variable $M$. I thought the calculation would be $$cov[X,Y]=E_M[cov[X,Y|M]] =E_M[E_{XY}[(X|M-E_X[X|M])(Y|M-E_Y[Y|M])]].$$ But since $X$ and $Y$ are conditionally independent, $cov[X,Y|M]=0$, and thus, $E_M[cov[X,Y|M]]=0$. Obviously this is false. Would replacing $E_X[X|M]$ with $E_M[E_X[X|M]]$ (and doing the same for $Y$) give me the correct calculation?

A simple yes or no answer will suffice, but I would be especially appreciative of a simple derivation of $cov[X,Y]$ in this situation.

Best Answer

In order to understand why the expression $\operatorname{Cov}(X,Y)=E_M\left[\operatorname{Cov}(X,Y|M)\right]$ is wrong write the definition of conditional covariance

$$\operatorname{Cov}(X,Y\mid M) = E(XY\mid M) - E(X\mid M)E(Y\mid M)$$

Now average over $M$

$$E_M\left[\operatorname{Cov}(X,Y\mid M)\right] = E_M\Big[ E(XY\mid M)\Big] - E_M\Big[ E(X\mid M)\cdot E(Y\mid M)\Big]$$

While $E_M\Big[ E(XY\mid M)\Big] = E(XY)$, the second term cannot be split into $E_M\Big[ E(X\mid M)\Big]\cdot E_M\Big[E(Y\mid M)\Big]$, and so it does not equal $E(X)E(Y)$.

And the reason it cannot be split is that what becomes independent conditional on $M$ are the random variables $\{X,Y\}$ and not the random variables $\big\{ E(X\mid M), E(Y\mid M)\big\}$.

Therefore $\operatorname{Cov}(X,Y)\neq E_M\left[\operatorname{Cov}(X,Y|M)\right]$. The correct expression is

$$\operatorname{Cov}(X,Y)=E_M[\operatorname{Cov}(X,Y \mid M)]+\operatorname{Cov}\big[E(X\mid M),E(Y\mid M)\big]$$

and you can find at Law of Total Covariance, how it is derived.