Suppose we have 2 normal distributions $X$ and $Y$ with mean $u_1$ and $u_2$ and variance $\sigma_1^2$ and $\sigma_2^2$; find $E[X\mid Y]$ and $\operatorname{Var}(X\mid Y)$.
I know
$$E[X\mid Y] = \mu_1 + \rho\sigma_1 \frac{Y – u_2}{\sigma_2} $$
and
$$\operatorname{Var}[X\mid Y] = \sigma_1 (1 – \rho^2)$$
but I can't prove it.
For $E[X\mid Y]$ I start with
$$E[X\mid Y] = \int_{-\infty}^{+\infty} x f_{X|Y}(x\mid y)\ dx$$
but that doesn't work because for calculating $f_{X\mid Y}(x\mid y)$ I need $f_{X,Y}(x,y)$ I don't have that.
Can any one help me?
Best Answer
Let us take for granted that $$\frac{X-\mu_1}{\sigma_1} = \rho \frac{Y-\mu_2}{\sigma_2} + \sqrt{1-\rho^2} Z \tag{$*$}$$ where $Z \sim N(0,1)$ is independent of $Y$. See the end of my answer for an explanation.
Then \begin{align} E[X \mid Y] &= \mu_1 + \sigma_1 E[(X-\mu_1)/\sigma_1 \mid Y] \\ &= \mu_1 + \sigma_1 \left( E[\rho (Y-\mu_2)/\sigma_2 \mid Y] + E[\sqrt{1-\rho^2} Z \mid Y] \right) \\ &= \mu_1 + \sigma_1 (\rho(Y - \mu_2) / \sigma_2) + \sqrt{1-\rho^2} E[Z] \\ &= \mu_1 + \rho \frac{\sigma_1}{\sigma_2} (Y-\mu_2). \end{align} Note that Ottavio Bartenor fixed a typo in your original expression for $E[X\mid Y]$.
Similarly, \begin{align} \text{Var}(X \mid Y) &= \sigma_1^2 \text{Var}((X-\mu_1)/\sigma_1 \mid Y) \\ &= \sigma_1^2 \text{Var}(\rho(Y-\mu_2)/\sigma_2 + \sqrt{1-\rho^2} Z \mid Y) \\ &= \sigma_1^2 \text{Var}(\sqrt{1-\rho^2} Z \mid Y) \\ &= \sigma_1^2 (1-\rho^2) \text{Var}(Z) \\ &= \sigma_1^2(1-\rho^2). \end{align} Note that the expression in your post has a typo.
Suppose $Y \sim N(\mu_2, \sigma_2^2)$ and $Z \sim N(0,1)$ are independent. Let $X$ satisfy the above equality ($*$). The claim is that $(X,Y)$ follows a bivariate normal distribution with parameters $\mu_1, \sigma_1, \mu_2 ,\sigma_2, \rho$.
You can check that $X$ has mean $\mu_1$ and variance $\sigma_1^2$. You can also check that the correlation between $X$ and $Y$ is $\rho$. You can also check that the marginal distribution of $X$ is normal, because it is a linear combination of independent normal random variables. Finally, to justify that $(X,Y)$ is jointly (bivariate) normal, you can appeal to an equivalent characterization of jointly normal distributions by noting that any linear combination of $X$ and $Y$ is normal because it can be written as a linear combination of the independent normal random variables $Y$ and $Z$