Suppose that $X_1, X_2, X_3$ are i.i.d. normal random variables with mean $0$ and variance $1$. And Suppose that $Z \sim N(1, 2^2)$ and is independent of all $X_i$. Define $Z_i = Z + X_i$ for $i = 1, 2, 3$. What is the distribution of the random vector $Z$?
I guess it is also normal but I don't know how exactly why. Any help would be appreciated.
Best Answer
The full answer to your question of why the answer is still normal requires appeal to a highly desirable property of the normal distribution: that it is a stable distribution. The theory of stable distributions requires characteristic equations and somewhat advanced, measure theoretic/real analytic arguments. Suffice it to say, most statisticians simply accept this fact, and henceforth just remember that linear combinations of normal variables are indeed normal.
In your case, your formulation is essentialy creating a multivariate normal vector with a gaussian "effect" added to it. Therefore, the resulting multivariate vector will again be a multivariate normal, but with a mean vector of $\mathbf{1}$ (since each X has mean 0 and Z has mean 1). However, the presence of the same Z in each term means that the resulting multivariate distribution is no longer a set of iid variables, since the common Z induces a correlation. As Dilip has pointed out, these covariances will be:
$cov(X_i+Z,X_j+Z)=cov(X_i,X_j)+cov(X_i,Z)+cov(Z,X_j)+cov(Z,Z)=\sigma^2_Z$ As all the other terms become zero due to independence.
Therefore, the correlation between any two elements of the resulting multivariate normal vector, ($X_i+Z,X_j+Z)$, will be $\rho = \frac{cov(X_i+Z,X_j+Z)}{\sqrt{\sigma_i^2+\sigma_Z^2}+ \sqrt{\sigma_j^2+\sigma_Z^2}}=\frac{4}{2\sqrt{5}}\approx .0.89$ Resulting in quite a strong linear relationship between the variables.