Chi-Square Distribution – Proving Squares of Normal RVs

chi-squared-testdistributionsmathematical-statisticsprobabilityself-study

I start with three independent random variables, $X_1, X_2, X_3$. They are each normally distributed with:

$$X_i \sim N(\mu_i, \sigma^2), i = 1, 2, 3.$$

I then have three transformations,

$$\eqalign{
Y_1 &= -X_1/\sqrt{2} + X_2/\sqrt{2} \cr
Y_2 &= -X_1/\sqrt{3} – X_2/\sqrt{3} + X_3/\sqrt{3} \cr
Y_3 &= X_1/\sqrt{6} + X_2/\sqrt{6} + 2X_3 / \sqrt{6} \cr
}$$

I am supposed to show that when $\mu_i = 0,$ $i = 1, 2, 3,$ $(Y_1^2 + Y_2^2 + Y_3^2)/\sigma^2 \sim \chi^2(3)$. I have also shown the transformations to preserve the independence, as the transformation matrix is orthogonal.

I have already shown that the expectations of $Y_1, Y_2, Y_3$ is 0 and their variances are all the same. Using the normal pdf, I have shown that:

$$Y_i^2 \sim \frac{1}{2\pi\sigma^2} \exp(-2x^2 / 2\sigma^2).$$

I thought about applying a substitution of $z = 2x^2 / \sigma^2$ to get the exponent into a similar form as the chi-square's $\exp(-x/2)$ form, but I'm stuck on what to do with the constants outside to get them to look similar. Could someone offer a hand?

Best Answer

We have $X_1\sim N(\mu_1,\sigma^2)$ and $X_2\sim N(\mu_2,\sigma^2)$, hence

$$EY_1=E(-X_1/\sqrt{2}+X_2/\sqrt{2})=-1/\sqrt{2}EX_1+1/\sqrt{2}EX_2=0$$

\begin{align*} EY_1^2&=E(-X_1/\sqrt{2}+X_2/\sqrt{2})^2\\\\ &=E(X_1/\sqrt{2})^2-2E(X_1X_2/2)+E(X_2/\sqrt{2})^2\\\\ &=1/2\sigma^2+1/2\sigma^2=\sigma^2 \end{align*}

Hence $Y_1\sim N(0,\sigma^2)$ since it is the linear combination of normal variables.

Similarly we get $Y_2\sim N(0,\sigma^2)$ and $Y_3\sim N(0,\sigma^2)$

Now

$$EY_1Y_2=1/\sqrt{6}E(X_1)^2-1/\sqrt{6}EX_2^2=0$$

and similarly $EY_2Y_3=EY_1Y_3=0$, hence $Y_1$, $Y_2$ and $Y_3$ are independent, since for normal variables independece coincided with zero correlation.

Having established that we have

$$(Y_1^2+Y_2^2+Y_3^2)/\sigma^2=\left(\frac{Y_1}{\sigma}\right)^2+\left(\frac{Y_2}{\sigma}\right)^2+\left(\frac{Y_3}{\sigma}\right)^2=Z_1^2+Z_2^2+Z_3^2$$,

where $Z_i=Y_i/\sigma$. Since $Y_i\sim N(0,\sigma^2)$, we have $Z_i\sim N(0,1)$.

We have showed that our quantity of interest is a sum of squares of 3 independent standard normal variables, which by definition is $\chi^2$ with 3 degrees of freedom.

As I've said in the comments you do not need to calculate the densities. If you on the other hand want to do that, your formula is wrong. Here is why. Denote by $G(x)$ distribution of $Y_1^2$ and $F(x)$ the distribution of $Y_1$. Then we have

$$G(x)=P(Y_1^2<x)=P(-\sqrt{x}<Y_1<\sqrt{x})=F(\sqrt{x})-F(-\sqrt{x})$$

Now the density of $Y_1^2$ is $G'(x)$, so

$$G'(x)=\frac{1}{2\sqrt{x}}(F'(\sqrt{x})+F'(-\sqrt{x})$$

We have that

$$F'(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{x^2}{\sigma^2}},$$

$$G'(x)=\frac{1}{\sigma\sqrt{2\pi x}}e^{-\frac{x}{2}}$$

If $\sigma^2=1$ we have a pdf of $\chi^2$ with one degree of freedom. (Note that for $Z_1$ instead of $Y_1$ the calculation is similar and $\sigma^2=1$ ) As @whuber pointed out, this is gamma distribution, and sums of independent gamma distributions is again gamma, the exact formula is provided in the wikipedia page.

Related Solutions

Mathematical Statistics – Are $U=\frac{2X_1^2}{(X_2+X_3)^2}$ and $V=\frac{2(X_2-X_3)^2}{2X_1^2+(X_2+X_3)^2}$ Independent?

Let $X=X_1,$ $Y=(X_2+X_3)/\sqrt2,$ and $Z=(X_2-X_3)/\sqrt2.$ As in your question, it is apparent that these are iid standard Normal. Moreover,

$$U = \frac{X^2}{Y^2}\ \text{ and }\ V = 2\frac{Z^2}{X^2 + Y^2}.$$

Consider, then, how $X/Y$ and $X^2+Y^2$ are related. The latter is the squared distance from the origin while the former is the cotangent of the angle $\Theta$ made by $(X,Y)$ to the $X$-axis. Because iid Normal variables are spherical, the angle and the distance are independent.

Recall that any (measurable) functions of independent variables are independent. Call these functions $f$ and $g.$ Beginning with the variables $\Theta$ and $(R, Z) = (\sqrt{X^2+Y^2}, Z),$ we have just seen $(\Theta,R,Z)$ are independent. Observing that we may express $V = 2Z^2/R^2 = g(R,Z)$ and $U = \cot^2(\Theta) = f(\Theta),$ we immediately see $(U,V)$ are independent.

In retrospect, it is evident $(\Theta, R, Z)$ is a cylindrical coordinate system for the original Cartesian coordinates $(X_1,X_2,X_3).$ Thus, if you prefer an explicitly rigorous, Calculus-based derivation, consider computing the joint distribution function in these cylindrical coordinates: it should separate into a term for $\theta$ and a term for $(r,z).$

BTW, this is a challenge for those of us who like to draw inspiration from simulations: both $U$ and $V$ have infinite means and, in even fairly large simulations, the $(U,V)$ scatterplot can look decidedly dependent. Here, for instance, is a log-log plot for 4,000 random $(U,V)$ pairs.

The lack of many extreme values of $U$ makes it look like $V$ tends to be high when $U$ is extreme.

Here's the R simulation code.

n <- 4e3
x <- rnorm(n)
y <- rnorm(n)
z <- rnorm(n)

u <- 2*x^2 / (y+z)^2
v <- 2*(y-z)^2 / (2*z^2 + (y+z)^2) 
plot(u,v, log="xy", col=gray(0, alpha=0.1), asp=1)

Best Answer

Related Solutions

Mathematical Statistics – Are $U=\frac{2X_1^2}{(X_2+X_3)^2}$ and $V=\frac{2(X_2-X_3)^2}{2X_1^2+(X_2+X_3)^2}$ Independent?

Related Question