I am trying to prove the following:
Let $(X_1,\dots,X_n)$ be a Gaussian vector with mean 0 and covariance matrix B. Find the distribution of $E(X_1\mid X_2,\dots,X_n).$
I know in general for two Gaussian r.v. $X_1$ and $X_2$ we can show that $f_{X_1|X_2} (x_1|x_2) = \frac{1}{\sigma_{X_1} \sqrt{2 \pi (1-\rho^2)}}\exp\frac{(-(x_{1}- \rho(\sigma_{X_1}/ \sigma_{X_2})x_2)^2)}{2 \sigma^{2}_{X_1}(1- \rho^{2})}$ where $\rho$ is the correlation and
$E(X_{1}|X_{2})=\int x_{1} f_{X_1|X_2} (x_1|x_2)dx_{1}. $
How can I generalize this for a gaussian random vector of size $n$? Can I conclude on the distribution of $E(X_1|X_2,…X_n)$ based on the form of $f_{X_1|X_2,\dots,X_n} (x_1|x_2,\dots,x_n)$? For example, in the case of 2 Gaussian r.v we produce a normal r.v with variance $2 \sigma^{2}_{X_1}(1- \rho^{2})$ and mean $\rho(\sigma_{X_1}/ \sigma_{X_2})x_2$.
Best Answer
Partition the column vector $X:=(X_1, X_2,\ldots, X_n)^T$ into subvectors $X_a$ and $X_b$: $$ X = \left(\begin{matrix}X_a\\X_b\end{matrix}\right) $$ (in your case $X_a=X_1$ is univariate) and correspondingly partition the mean vector $\mu$ and covariance matrix $\Sigma$ of $X$: $$ \mu = \left(\begin{matrix}\mu_a\\ \mu_b\end{matrix}\right) $$ $$ \Sigma=\left(\begin{matrix}\Sigma_{a,a}&\Sigma_{a,b}\\\Sigma_{b,a}&\Sigma_{b,b}\end{matrix}\right)$$ Assume that $\Sigma_{b,b}$ is invertible. To find the conditional expectation $E(X_a\mid X_b)$, first find a matrix $C$ of constants such that $Z:=X_a- C X_b$ is uncorrelated with $X_b$. For this to be true we demand $$ 0= \operatorname{cov} (Z, X_b)=\operatorname{cov} (X_a - CX_b, X_b)=\Sigma_{a,b}-C\Sigma_{b,b}, $$ which yields $$ C=\Sigma_{a,b}\Sigma_{b,b}^{-1}. $$ Therefore $$ \begin{align} E(X_a\mid X_b)&=E(Z + C X_b\mid X_b)\\ &=E(Z\mid X_b) + CX_b\\ &\stackrel{(*)}=E(Z) + CX_b\\ &= E(X_a) + C(X_b - E(X_b))\\ &= \mu_a + \Sigma_{a,b}\Sigma_{b,b}^{-1}(X_b - \mu_b),\tag1 \end{align} $$ where in (*) we use the fact that $Z$ and $X_b$ are uncorrelated multivariate Gaussian random vectors and therefore are independent. From (1) we see that $E(X_a\mid X_b)$ is a linear combination of the elements of the Gaussian vector $X_b$, and therefore also is Gaussian.