Distribution of ratio of sample variances

probabilityprobability theorystatistical-inferencestatistics

I have two independent zero mean populations (not necessarily normally distributed) and I am trying to estimate the ratio of variances of the two population. I have i.i.d. sample from each population of size $n$.
I do know that sample variance converges to population variance almost surely, so by Slutsky theorem, ratio of variances will converge to ratio population variances almost surely. Thus, ratio of sample variances is a consistent estimator.

But what can I say about it's asymptotic distribution, of ratio of sample variances without knowing the distribution of the population or assuming it to be normal?
I do know when population distribution is normal then it follows $F$ distribution with $(n-1,n-1)$ degrees of freedom. But what about the general case?

Thanks in Advance

Best Answer

Let

$$ S_X^2 = \frac{1} {n} \sum_{i=1}^n X_i^2, S_Y^2 = \frac{1} {n} \sum_{i=1}^n Y_i^2$$

be the sample variance estimator of the variances $\sigma_X^2$ and $\sigma_Y^2$ respectively.

By the bivariate CLT,

$$ \sqrt{n} \left(\begin{bmatrix} S_X^2 \\ S_Y^2 \end{bmatrix} - \begin{bmatrix} \sigma_X^2 \\ \sigma_Y^2 \end{bmatrix} \right) \stackrel {d} {\to} \mathcal{N} \begin{bmatrix} Var[X_1] & 0 \\ 0 & Var[Y_1] \end{bmatrix} $$

where

$$ Var[X_1] = E[X_1^4] - E[X_1^2]^2 = E[X_1^4] - \sigma_X^4$$ $$ Var[Y_1] = E[Y_1^4] - E[Y_1^2]^2 = E[Y_1^4] - \sigma_Y^4$$

Next we will need the bivariate Delta method to obtain the asymptotic distribution of $\displaystyle \frac {S_X^2} {S_Y^2}$

See, e.g. https://en.wikipedia.org/wiki/Taylor_expansions_for_the_moments_of_functions_of_random_variables

Consider $\displaystyle f(x, y) = \frac {x} {y}$. Then

$$ \frac {\partial f} {\partial x} = \frac {1} {y}, ~~ \frac {\partial f} {\partial y} = - \frac {x} {y^2}, ~~ \frac {\partial^2 f} {\partial x^2} = 0, ~~ \frac {\partial^2 f} {\partial y^2} = \frac {2x} {y^3}, ~~ \frac {\partial^2 f} {\partial x \partial y} = - \frac {1} {y^2}$$

We Taylor Expand $\displaystyle \frac {S_X^2} {S_Y^2} = f(S_X^2, S_Y^2)$ about the mean $(\sigma_X^2, \sigma_Y^2)$ up to the second order:

$$ \begin{align} f(S_X^2, S_Y^2) \approx &~ f(\sigma_X^2, \sigma_Y^2) + \left.\frac {\partial f} {\partial x}\right|_{(\sigma_X^2, \sigma_Y^2)} (S_X^2 - \sigma_X^2) + \left.\frac {\partial f} {\partial y}\right|_{(\sigma_X^2, \sigma_Y^2)} (S_Y^2 - \sigma_Y^2) \\ &~ + \frac {1} {2} \left.\frac {\partial^2 f} {\partial x^2}\right|_{(\sigma_X^2, \sigma_Y^2)}(S_X^2 - \sigma_X^2)^2 + \frac {1} {2} \left.\frac {\partial^2 f} {\partial y^2}\right|_{(\sigma_X^2, \sigma_Y^2)}(S_Y^2 - \sigma_Y^2)^2 \\ &~ + \left.\frac {\partial^2 f} {\partial x \partial y}\right|_{(\sigma_X^2, \sigma_Y^2)}(S_X^2 - \sigma_X^2)(S_Y^2 - \sigma_Y^2) \\ \Rightarrow \frac {S_X^2} {S_Y^2} \approx &~ \frac {\sigma_X^2} {\sigma_Y^2} + \frac {1} {\sigma_Y^2} (S_X^2 - \sigma_X^2) - \frac {\sigma_X^2} {\sigma_Y^4} (S_Y^2 - \sigma_Y^2) \\ &~ + 0 + \frac {\sigma_X^2} {\sigma_Y^6} (S_Y^2 - \sigma_Y^2)^2 - \frac {1} {\sigma_Y^4} (S_X^2 - \sigma_X^2)(S_Y^2 - \sigma_Y^2) \end{align} $$

Taking expectation,

$$ E\left[\frac {S_X^2} {S_Y^2}\right] \approx \frac {\sigma_X^2} {\sigma_Y^2} + 0 + 0 + \frac {\sigma_X^2} {\sigma_Y^6} Var[S_Y^2] - 0 = \frac {\sigma_X^2} {\sigma_Y^2} + \frac {\sigma_X^2(E[Y_1^4] - \sigma_Y^4)} {n\sigma_Y^6} $$

and variance (with first order only) $$ Var\left[\frac {S_X^2} {S_Y^2}\right] \approx 0 + \frac {1} {\sigma_Y^4}Var[S_X^2] + \frac {\sigma_X^4} {\sigma_Y^8}Var[S_Y^2] = \frac {E[X_1^4] - \sigma_X^4} {n\sigma_Y^4} + \frac {\sigma_X^4(E[Y_1^4] - \sigma_Y^4)} {n\sigma_Y^8}$$

So by Delta method, collecting the terms,

$$ \sqrt{n} \left(\frac {S_X^2} {S_Y^2} - \frac {\sigma_X^2} {\sigma_Y^2} \right) \stackrel {d} {\to} \mathcal{N} \left(0, \frac {E[X_1^4] - \sigma_X^4} {\sigma_Y^4} + \frac {\sigma_X^4(E[Y_1^4] - \sigma_Y^4)} {\sigma_Y^8} \right)$$

The second order term here just serve as a correction term providing higher accuracy if needed.

Related Solutions

Sampling Distribution of Variances

Both look wrong to me. Your error is that the sample variance is proportional to a chi-square distribution with $4$ degrees of freedom, and you have missed this proportionality of $4$ or $5$ depending on how you calculate sample variances; you should try $4 \times \frac{10}{15}$ or $5 \times \frac{10}{15}$ rather than just $\frac{10}{15}$ as the value you are testing

If $X_1,X_2,\ldots,X_n$ were i.i.d. $\sim N(\mu,1)$, then I would have thought that $$\sum_i (X_i-\bar X)^2 = (n-1)\times \frac{1}{n-1}\sum_i (X_i-\bar X)^2 \sim \chi^2_{n-1}$$ so using R, I would have thought you would answer this question with

pchisq((5-1) * 10/15, df=5-1)
[1] 0.38494

A simulation seems to produce a close figure:

library(matrixStats)
set.seed(1)
cases     <- 1000000
n         <- 5
popvar    <- 15
critvalue <- 10
matdat <- matrix(rnorm(cases*n, mean=0, sd=sqrt(popvar)), ncol=n)
samplevars <- rowVars(matdat)
mean(samplevars < critvalue)
[1] 0.384494

But if the book's definition of sample variance is instead $\frac{1}{n}\sum_i (X_i-\bar X)^2$ rather than R's $\frac{1}{n-1}\sum_i (X_i-\bar X)^2$ then these would indeed become closer to $0.50$

pchisq(5 * 10/15, df=5-1)
[1] 0.4963317

samplevars_n <- rowVars(matdat) * (n-1)/n
mean(samplevars_n < critvalue)
[1] 0.495974

How population size affects the variance of the distribution of sample mean

If your sample is without replacement from a finite population size $N$, then the variance of the sample mean is not $\dfrac{\sigma^2}{n}$ but $\dfrac{N-n}{N-1}\dfrac{\sigma^2}{n}$

So in your example of $n=100$, $N_1=120$ and $N_2=120000$, then instead of a variance of $\dfrac{\sigma^2}{100}$, we would get $\dfrac{\sigma^2}{595}$ for the smaller population and about $\dfrac{\sigma^2}{100.083}$ for the variance population, with the variance for the smaller population substantially smaller, in line with your intuition

Best Answer

Related Solutions

Sampling Distribution of Variances

How population size affects the variance of the distribution of sample mean

Related Question