Solved – Expectation of product of Gaussian random variables

expected valuenormal distributionrandom variable

Say we have two Gaussian random vectors $p(x_1) = N(0,\Sigma_1), p(x_2) = N(0,\Sigma_2)$, is there a well known result for the expectation of their product $E[x_1x_2^T]$ without assuming independence?

Best Answer

Yes, there is a well-known result. Based on your edit, we can focus first on individual entries of the array $E[x_1 x_2^T]$. Such an entry is the product of two variables of zero mean and finite variances, say $\sigma_1^2$ and $\sigma_2^2$. The Cauchy-Schwarz Inequality implies the absolute value of the expectation of the product cannot exceed $|\sigma_1 \sigma_2|$. In fact, every value in the interval $[-|\sigma_1 \sigma_2|, |\sigma_1 \sigma_2|]$ is possible because it arises for some binormal distribution. Therefore, the $i,j$ entry of $E[x_1 x_2^T]$ must be less than or equal to $\sqrt{\Sigma_{1_{i,i}} \Sigma_{2_{j,j}}}$ in absolute value.

If we now assume all variables are normal and that $(x_1; x_2)$ is multinormal, there will be further restrictions because the covariance matrix of $(x_1; x_2)$ must be positive semidefinite. Rather than belabor the point, I will illustrate. Suppose $x_1$ has two components $x$ and $y$ and that $x_2$ has one component $z$. Let $x$ and $y$ have unit variance and correlation $\rho$ (thus specifying $\Sigma_1$) and suppose $z$ has unit variance ($\Sigma_2$). Let the expectation of $x z$ be $\alpha$ and that of $y z$ be $\beta$. We have established that $|\alpha| \le 1$ and $|\beta| \le 1$. However, not all combinations are possible: at a minimum, the determinant of the covariance matrix of $(x_1; x_2)$ cannot be negative. This imposes the non-trivial condition

$$1-\alpha ^2-\beta ^2+2 \alpha \beta \rho -\rho ^2 \ge 0.$$

For any $-1 \lt \rho \lt 1$ this is an ellipse (along with its interior) inscribed within the $\alpha, \beta$ square $[-1, 1] \times [-1, 1]$.

To obtain further restrictions, additional assumptions about the variables are necessary.

Plot of the permissible region $(\rho, \alpha, \beta)$

alt text