Probability – Proving Independence of Two Random Variables with F-Distribution

f distributionindependenceprobabilityself-study

The following is an exercise problem from Hogg, McKean and Craig's book titled "Introduction to Mathematical Statistics" that I am working on.

3.6.16. Let $X_1$, $X_2$, and $X_3$ be three independent chi-square variables with $r_1$, $r_2$, and $r_3$ degrees of freedom, respectively.

(a) Show that $Y_1 = X_1/X_2$ and $Y_2 = X_1 + X_2$ are independent and that $Y_2$ is
$\chi^2(r_1 + r_2)$.

(b) Deduce that
$$\cfrac{X_1 /r_1}{X_2/r_2} \ \ \textrm{ and } \ \ \cfrac{X_3 /r_3}{(X_1 + X_2)/(r_1 + r_2)}$$ are independent F-variables.

My attempt:

(a) The inverse of the transformations is given by $X_1 = Y_1Y_2/(1+Y_1)$ and $X_2=Y_2/(1+Y_1)$ and the Jacobian is the determinant of
$$ \begin{bmatrix} \frac{\partial x_1}{\partial y_1} & \frac{\partial x_1}{\partial y_2} \\ \frac{\partial x_2}{\partial y_1} & \frac{\partial x_2}{\partial y_2} \end{bmatrix} = \begin{bmatrix} y_2/(1+y_1)^2 & y_1/(1+y_1) \\ -y_2/(1+y_1)^2 & 1/(1+y_1) \end{bmatrix} $$ which ends up being $y_2/(1+y_1)^2$. The joint pdf of $Y_1$ and $Y_2$ is then given by

\begin{align}f_{Y_1,Y_2}(y_1,y_2) &= \cfrac{y_2}{(1+y_1)^2} \left( \cfrac{y_1y_2}{1+y_1} \right)^{\left(\frac{r_1}{2}-1\right)}\cfrac{e^{-\left(\frac{y_1y_2}{2(1+y_1)}\right)}}{2^{\frac{r_1}{2}}\Gamma(\frac{r_1}{2})}\left( \cfrac{y_2}{1+y_1} \right)^{\left(\frac{r_2}{2}-1\right)}\cfrac{e^{-\left(\frac{y_2}{2(1+y_1)}\right)}}{2^{\frac{r_2}{2}}\Gamma(\frac{r_2}{2})} \\ &= \cfrac{1}{2^{\left(\frac{r_1+r_2}{2}\right)}\Gamma(\frac{r_1}{2})\Gamma(\frac{r_2}{2})} \times
\left[ \frac{y_1^{\left( \frac{r_1}{2}-1\right)}}{(1+y_1)^{\left(\frac{r_1+r_2}{2} \right)}}\right]
\times\left[y_2^{\left(\frac{r_1+r_2}{2}-1\right)}e^{-\left(\frac{y_2}{2}\right)}\right]\end{align}.

With that, the pdf nicely splits into a product of a function in $y_1$ alone, and that of $y_2$ alone thereby letting us to conclude that $Y_1$ and $Y_2$ are independent. We need to be careful about the support before we conclude that but as the support is that of positive real numbers for both of them, we should not have a problem on that front.

(b) Let $Y_3 = X_3/(X_1+X_2)$. We need to show that $Y_1$ and $Y_3$ are independent. I will spare the details but taking an approach similar to the one above, we get the Jacobian to be $y_2^2/(1+y_1)^2$ and the joint pdf to be

\begin{align}f_{Y_1,Y_2,Y_3}(y_1,y_2,y_3) &= \cfrac{1}{2^{\left(\frac{r_1+r_2+r_3}{2}\right)}\Gamma(\frac{r_1}{2})\Gamma(\frac{r_2}{2})\Gamma(\frac{r_3}{2})} \times
\left[ \frac{y_1^{\left( \frac{r1}{2}-1\right)}y_3^{\left(\frac{r_3}{2}-1\right)}}{(1+y_1)^{\left(\frac{r_1+r_2}{2} \right)}}\right]\times
\left[y_2^{\left(\frac{r_1+r_2+r_3}{2}-1\right)}e^{-\left(\frac{y_2(1+y_3)}{2}\right)}\right].\end{align}

Clearly, $Y_1$, $Y_2$ and $Y_3$ are not jointly independent. However, when we integrate out $y_2$ over its support to obtain the marginal pdf $f_{Y_1,Y_3}(y_1,y_3)$, the expression in the second square braces integrates to some kind of Gamma function in $y_3$ along with possibly another factor involving $2$ rasied to a power of $y_3$, which will let us conclude that $Y_1$ and $Y_3$ are independent.

This still lacks details but I think I have explained my main thought process above. Here are my questions to you.

Do you find any error in this thought process?
More importantly, is there a more elegant/sleek solution to part (b) that uses part (a) and some other result? Please note that $Y_1$, $Y_2$, and $Y_3$ are not mutually independent. If they were, part (b) would have been quite trivial.

I am asking this question mainly because in part (b) he starts with "Deduce that …" by which I feel like there might be a one liner to deduce that $Y_1$ and $Y_3$ are independent. Is there a simpler solution? Please let me know.

Best Answer

Your answer to (a) is correct.

For part (b), the two random variables are $F$-distributed by construction. We could prove that they're independent by establishing joint independence of $Y_1, Y_2, X_3$.

I believe you can do this via the joint moment-generating function. We have $$ M_{Y_1,Y_2,X_3}(s,t,u) = \mathbb{E}[\exp(sY_1+tY_2+uX_3)]=\mathbb{E}[\exp(uX_3) \mathbb{E}[\exp(sY_1+tY_2|X_3)] ] $$ We can drop the conditioning on $X_3$ in the inner expectation, as $ (Y_1, Y_2) $ are constructed from $(X_1,X_2)$ only. This gives $$ \mathbb{E}[\exp(sY_1+tY_2|X_3)]=\mathbb{E}[\exp(sY_1+tY_2)]=M_{Y_1,Y_2}(s,t)=M_{Y_1}(s)M_{Y_2}(t), $$ by independence of $Y_1,Y_2$.

Putting it all together, we have $$ M_{Y_1,Y_2,X_3}(s,t,u)=\mathbb{E}[\exp(uX_3) M_{Y_1}(s)M_{Y_2}(t)] =M_{Y_1}(s)M_{Y_2}(t)M_{X_3}(u) $$ and we have joint independence.

Related Solutions

Chi-Square Distribution – Proving Squares of Normal RVs

We have $X_1\sim N(\mu_1,\sigma^2)$ and $X_2\sim N(\mu_2,\sigma^2)$, hence

$$EY_1=E(-X_1/\sqrt{2}+X_2/\sqrt{2})=-1/\sqrt{2}EX_1+1/\sqrt{2}EX_2=0$$

\begin{align*} EY_1^2&=E(-X_1/\sqrt{2}+X_2/\sqrt{2})^2\\\\ &=E(X_1/\sqrt{2})^2-2E(X_1X_2/2)+E(X_2/\sqrt{2})^2\\\\ &=1/2\sigma^2+1/2\sigma^2=\sigma^2 \end{align*}

Hence $Y_1\sim N(0,\sigma^2)$ since it is the linear combination of normal variables.

Similarly we get $Y_2\sim N(0,\sigma^2)$ and $Y_3\sim N(0,\sigma^2)$

Now

$$EY_1Y_2=1/\sqrt{6}E(X_1)^2-1/\sqrt{6}EX_2^2=0$$

and similarly $EY_2Y_3=EY_1Y_3=0$, hence $Y_1$, $Y_2$ and $Y_3$ are independent, since for normal variables independece coincided with zero correlation.

Having established that we have

$$(Y_1^2+Y_2^2+Y_3^2)/\sigma^2=\left(\frac{Y_1}{\sigma}\right)^2+\left(\frac{Y_2}{\sigma}\right)^2+\left(\frac{Y_3}{\sigma}\right)^2=Z_1^2+Z_2^2+Z_3^2$$,

where $Z_i=Y_i/\sigma$. Since $Y_i\sim N(0,\sigma^2)$, we have $Z_i\sim N(0,1)$.

We have showed that our quantity of interest is a sum of squares of 3 independent standard normal variables, which by definition is $\chi^2$ with 3 degrees of freedom.

As I've said in the comments you do not need to calculate the densities. If you on the other hand want to do that, your formula is wrong. Here is why. Denote by $G(x)$ distribution of $Y_1^2$ and $F(x)$ the distribution of $Y_1$. Then we have

$$G(x)=P(Y_1^2<x)=P(-\sqrt{x}<Y_1<\sqrt{x})=F(\sqrt{x})-F(-\sqrt{x})$$

Now the density of $Y_1^2$ is $G'(x)$, so

$$G'(x)=\frac{1}{2\sqrt{x}}(F'(\sqrt{x})+F'(-\sqrt{x})$$

We have that

$$F'(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{x^2}{\sigma^2}},$$

$$G'(x)=\frac{1}{\sigma\sqrt{2\pi x}}e^{-\frac{x}{2}}$$

If $\sigma^2=1$ we have a pdf of $\chi^2$ with one degree of freedom. (Note that for $Z_1$ instead of $Y_1$ the calculation is similar and $\sigma^2=1$ ) As @whuber pointed out, this is gamma distribution, and sums of independent gamma distributions is again gamma, the exact formula is provided in the wikipedia page.

Mathematical Statistics – Are $U=\frac{2X_1^2}{(X_2+X_3)^2}$ and $V=\frac{2(X_2-X_3)^2}{2X_1^2+(X_2+X_3)^2}$ Independent?

Let $X=X_1,$ $Y=(X_2+X_3)/\sqrt2,$ and $Z=(X_2-X_3)/\sqrt2.$ As in your question, it is apparent that these are iid standard Normal. Moreover,

$$U = \frac{X^2}{Y^2}\ \text{ and }\ V = 2\frac{Z^2}{X^2 + Y^2}.$$

Consider, then, how $X/Y$ and $X^2+Y^2$ are related. The latter is the squared distance from the origin while the former is the cotangent of the angle $\Theta$ made by $(X,Y)$ to the $X$-axis. Because iid Normal variables are spherical, the angle and the distance are independent.

Recall that any (measurable) functions of independent variables are independent. Call these functions $f$ and $g.$ Beginning with the variables $\Theta$ and $(R, Z) = (\sqrt{X^2+Y^2}, Z),$ we have just seen $(\Theta,R,Z)$ are independent. Observing that we may express $V = 2Z^2/R^2 = g(R,Z)$ and $U = \cot^2(\Theta) = f(\Theta),$ we immediately see $(U,V)$ are independent.

In retrospect, it is evident $(\Theta, R, Z)$ is a cylindrical coordinate system for the original Cartesian coordinates $(X_1,X_2,X_3).$ Thus, if you prefer an explicitly rigorous, Calculus-based derivation, consider computing the joint distribution function in these cylindrical coordinates: it should separate into a term for $\theta$ and a term for $(r,z).$

BTW, this is a challenge for those of us who like to draw inspiration from simulations: both $U$ and $V$ have infinite means and, in even fairly large simulations, the $(U,V)$ scatterplot can look decidedly dependent. Here, for instance, is a log-log plot for 4,000 random $(U,V)$ pairs.

The lack of many extreme values of $U$ makes it look like $V$ tends to be high when $U$ is extreme.

Here's the R simulation code.

n <- 4e3
x <- rnorm(n)
y <- rnorm(n)
z <- rnorm(n)

u <- 2*x^2 / (y+z)^2
v <- 2*(y-z)^2 / (2*z^2 + (y+z)^2) 
plot(u,v, log="xy", col=gray(0, alpha=0.1), asp=1)

Best Answer

Related Solutions

Chi-Square Distribution – Proving Squares of Normal RVs

Mathematical Statistics – Are $U=\frac{2X_1^2}{(X_2+X_3)^2}$ and $V=\frac{2(X_2-X_3)^2}{2X_1^2+(X_2+X_3)^2}$ Independent?

Related Question