[Math] Covariance and conditional probability

covarianceprobability

This may be completely trivial, but wanted to understand the following better: Let's say we know the covariance of two random variables $X, Y$ – is there anything intelligent we can say about the expected value of $X$, given an observation $Y = y$?

I.e. what (if any) is the relationship of $cov(X,Y)$ and $\mathbb{E}(X|Y = y)$?

If required, we may assume that we know $\mathbb{E}X=\mu$ and $\mathbb{E}{Y}=\gamma$.

Best Answer

$\newcommand{\E}{\operatorname{\mathbb{E}}}$ $\newcommand{\Cov}{\operatorname{\mathbb{Cov}}}$ $\newcommand{\Pr}{\operatorname{\mathbb{Pr}}}$ Just this: $$\begin{align} \E(XY) & = \sum_{\forall y} y\E(X\mid Y=y)\Pr(Y=y) & = \int_{\bf Y} y \E(X\mid Y=y)f_Y(y)\operatorname{d}y \\ \\ \Cov(X,Y) & = \E(XY)-\E(X)\E(Y) \\ & = \underbrace{-\mu\gamma + \sum_{\forall y} y\E(X\mid Y=y)\Pr(Y=y)}_{\text{discrete random variable}} & = \underbrace{-\mu\gamma + \int_{\bf Y} y \E(X\mid Y=y)f_Y(y)\operatorname{d}y}_{\text{continuous random variable}} \end{align}$$

Related Question