Consider an $n$-dimensional normal random vector $\mathbf X:= (X_1, \dots, X_n)^T$ with mean $\mathbf 0$ and covariance matrix $\mathbf \Sigma$. Now define a new random vector $\mathbf Y:= (a_1X_1, \dots, a_nX_n)^T$, where $a_1, \dots, a_n$ are distinct constants. What is the distribution of $\mathbf Y$, please? I vaguely remember that $\mathbf Y$ should still be a normal random vector with mean $\mathbf 0$. However, I cannot figure out its covariance matrix. Could anyone help me, please? Thank you!
[Math] Variance of Transformed Random Vectors
normal distributionprobability distributionsprobability theoryself-learning
Related Solutions
We find the mean of $\mathbf{y}$ by using the fact that $\mathbb{E}\{\}$ is a linear operator.
$$ \mathbf{\bar{y}} = \mathbb{E}\{\mathbf{y}\} = \mathbb{E}\{\mathbf{A}\mathbf{x}+\mathbf{b}\} = \mathbf{A}\mathbb{E}\{\mathbf{x}\}+\mathbf{b} = \mathbf{A}\mathbf{\bar{x}}+\mathbf{b} $$
Then we find covariance of
$$ \begin{array}{rcl} \mathbf{C_y} & \triangleq & \mathbb{E}\{(\mathbf{y}-\mathbf{\bar{y}})(\mathbf{y}-\mathbf{\bar{y}})^\top\} \\ & = & \mathbb{E} \Big\{ \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] ^\top \Big\} \\ & = & \mathbb{E} \Big\{ \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] ^\top \Big\} \\ & = & \mathbb{E} \Big\{ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^\top \mathbf{A}^\top \Big\} \\ & = & \mathbf{A} \mathbb{E} \Big\{ (\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^\top \Big\} \mathbf{A}^\top \\ & = & \mathbf{A}\mathbf{C_x}\mathbf{A}^\top \end{array} $$
Then, $\mathbf{y}$ is defined as,
$$ \mathbf{y} \sim \mathcal{N}(\mathbf{A}\mathbf{\bar{x}+\mathbf{b}, \mathbf{A}\mathbf{C_x}\mathbf{A}^\top}) $$
That is,
$$ f_\mathbf{Y}(\mathbf{y)} = {1 \over \sqrt{\lvert2\pi\mathbf{A}\mathbf{C_x}\mathbf{A}^\top\rvert}} \exp\left(- {1 \over 2} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big]^\top (\mathbf{A}\mathbf{C_x}\mathbf{A}^\top)^{-1} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big] \right) $$
The easiest way to convince yourself that the statement from Wikipedia you highlighted above is true is by use of the characteristic function. As we know it is defined as follows: \begin{equation} \kappa_\vec{X}(\vec{k}) := E\left[ e^{\imath \vec{k} \cdot \vec{X}} \right] \end{equation}
Now in case $\vec{X}$ is multivariate normal with means $\vec{\mu}$ and covariances ${\bf \Sigma}$,i.e. $\vec{X} \sim N(\vec{\mu},{\bf \Sigma})$ then from https://en.wikipedia.org/wiki/Multivariate_normal_distribution we have: \begin{equation} \kappa_\vec{X}(\vec{k}) = e^{\imath \vec{\mu}^T \cdot \vec{k} - \frac{1}{2} \vec{k}^T \cdot {\bf \Sigma} \cdot \vec{k}} \end{equation}
Now to marginalize in $\vec{k}$-space is easy. All we need to do is to set the respective components, i.e. those corresponding to variables we want to marginalize over in real space, of the $\vec{k}$-vector to zero. therefore we take ${\vec{k}_1} := \left( 0,\cdots 0, k_j, 0, \cdots, 0 \right)$ where the non-zero element sits at the jth position. Then clearly we have: \begin{equation} \kappa_\vec{X}(\vec{k}_1) = e^{\imath \mu_j k_j - \frac{1}{2} k_j {\bf \Sigma}_{j,j} k_j} \end{equation} Now in order to get the marginal distribution all we need to do is to invert the Fourier transform. We have: \begin{eqnarray} \rho^{(marg)}_j(x_j) &:=& \frac{1}{2\pi} \int\limits_{-\infty}^{\infty} e^{-\imath k_j x_j} \kappa_\vec{X}(\vec{k}_1) d k_j \\ &=& \frac{1}{2\pi} \int\limits_{\mathbb R} e^{-\imath k_j (x_j-\mu_j) - \frac{1}{2} {\bf \Sigma}_{j,j} k_j^2} dk_j \\ &=&e^{-\frac{1}{2} \frac{(x_j-\mu_j)^2}{{\bf \Sigma}_{j,j}} } \cdot \frac{1}{2\pi} \cdot \int\limits_{\mathbb R} e^{-\frac{1}{2} {\bf \Sigma}_{j,j} k_j^2 }dk_j \\ &=&\frac{1}{\sqrt{2 \pi {\bf \Sigma}_{j,j}}} e^{-\frac{1}{2} \frac{(x_j-\mu_j)^2}{{\bf \Sigma}_{j,j}} } \end{eqnarray} as expected.
Best Answer
You have $$Y=Diag(a_1,\dots,a_n)(X_1,\dots,X_n)^T$$
So $Y$ is normally distributed as a linear transform of a normally distributed vector.
Now you have the following properties for random vector $X$ and a (non-random) matrix $A$: $$E(AX)=AE(X)$$ This implies that $E(Y)=0$. $$V(AX)=AV(X)A^T$$ From this you can find the covariance matrix of $Y$. You can also find it "by hand" by calculating $Cov(Y_i,Y_j)$ for $1\leq i,j\leq n$.