If $X_1$, $X_2$, and $X_3$ are independent normally-distributed random variables, and $a_1, a_2$ are constants, does it follow that $(a_1 X_1 + a_2 X_2)$ and $X_3$ are independent? It seem obvious, but I can't easily think of a proof.
Independence of linear combinations of (normal) random variables
probabilityprobability distributions
Best Answer
If random variables $X_1, X_2, X_3$ are mutually independent (not just pairwise independent), then the $\sigma$-algebras $\Sigma_1, \Sigma_2, \Sigma_3$ they generate are independent. Normality is not needed. Since $a_1 X_1 + a_2 X_2$ is measurable with respect to the $\sigma$-algebra generated by $\Sigma_1$ and $\Sigma_2$, and this and $\Sigma_3$ are independent, $a_1 X_1 + a_2 X_2$ and $X_3$ are independent.
On the other hand, if $X_1$, $X_2$ and $X_3$ are only assumed to be pairwise independent, it's not true. A standard example is this:
Let $X_1, X_2, Y$ be jointly normal random variables with means $0$ and covariance matrix $I$. Let $$ X_3 = \cases{Y & if $X_1 X_2 Y \ge 0$\cr -Y & otherwise} $$ Then $X_1$, $X_2$ and $X_3$ are standard normal random variables which are pairwise independent, but $X_1 X_2 X_3 \ge 0$.
Now $X_1 + X_2$ and $X_3$ are not independent. In fact, consider the covariance of $(X_1 + X_2)^2$ and $X_3$. $$\text{Cov}((X_1+X_2)^2, X_3) = \text{Cov}(X_1^2, X_3) + \text{Cov}(X_2^2, X_3) + 2\text{Cov}(X_1 X_2, X_3)$$ The first two terms are $0$ because of pairwise independence, but the third term is positive because $X_1 X_2 X_3 > 0$ with probability $1$.