[Math] How to find the Type 2 Error of an F Test for equality of variances

statistics

Two random samples are drawn from Normal distributions. Consider an $\alpha$ = 0.01 test of

$H_0: \sigma_1=\sigma_2$ against $H_a: \sigma_1 > \sigma_2$

a) Suppose $n_1$ = $n_2$ = 12.

Determine $\beta$ when $\sigma_1$/$\sigma_2$=2.

I am confused on how to find this, I know how to find the type II error with other distributions but the F I find difficult.

Best Answer

You have a sample $X_1, X_2, \dots, X_{12}$ sampled at random from $\mathsf{Norm}(\mu_x, \sigma_x)$ and an independent sample $Y_1, Y_2, \dots, Y_{12}$ sampled at random from $\mathsf{Norm}(\mu_y, \sigma_y)$.

Let $\psi = \sigma_x^2/\sigma_y^2.$ You wish to test $H_0: \psi=1$ against $H_a: \psi > 1$ at level $\alpha = 0.01.$

Under $H_0: \psi = 1,$ you have the ratio of the sample variances $R = S_x^2/S_y^2 \sim \mathsf{F}(11,11)$ and you will reject $H_0$ if $R > c,$ where the critical value $c$ cuts 1% of the probability from the upper tail of $\mathsf{F}(11,11)$. You can find $c = 4.462$ from printed tables of the F-distribution or using software; the computation in R statistical software is shown below.

c = qf(.99, 11, 11);  c
## 4.462436

Roughly and intuitively, the observed variance ratio $R = S_x^2/S_y^2$ has to be above 4 in order to reject $H_0.$ This will happen rarely if $\psi = \sigma_x^2/\sigma_y^2 = 1.$ But you want to know the probability of rejection if $\sigma_x = 2\sigma_y$ so that $\psi = 4.$ In that case there should be a reasonable chance that the variance ratio exceeds $c$ and you can reject $H_0$. The probability of Type II error is the probability that you do not reject $H_0$ in these circumstances.

In general, $\frac{S_x^2/\sigma_x^2}{S_y^2/\sigma_y^2} = \frac{S_x^2}{S_y^2}/\psi \sim \mathsf{F}(n_x-1,n_y - 1).$ Thus, if $\psi = 4,$ then the probability of rejection is $P(R \ge c/4) = 0.4296$ and the probability of Type II Error is $P(R < c/4) = 0.5704.$

1 - pf(c/4, 11, 11)
## 0.4296341

In R, it is easy to make a 'power curve', plotting the probability of rejection against $\psi.$ Notice that the power of rejection increases as $\psi$ increases (that is, as the population variances become more different). In the plot below, red lines emphasize the power against the alternative $H_a: \psi = 4.$

enter image description here

psi = seq(1, 6, by=.01);  c = qf(.99,11,11)
p.rej = 1 - pf(c/psi,11,11)
plot(psi, p.rej, type="l", lwd=2, main="Power Curve for Right-Sided F Test")
  abline(v=1, col="green2"); abline(h=0, col="green2")
  lines(c(1,4,4), c(.43, .43, 0), col="red")

Finally, we simulate the power for $m = 10^6$ pairs of samples, each of size $n = 12$ from populations $\mathsf{Norm}(20, 2)$ and $\mathsf{Norm}(25, 1),$ respectively. [The means are not relevant, and $\psi = 2^2 = 4.$] For each of the $m$ pairs we determine whether $H_0$ is rejected at the 1% level, using $c$ as the critical value. As anticipated from our power computation above, the incorrect $H_0$ was rejected for about 43% of the simulated pairs. (A second run gave essentially the same result.)

set.seed(1234)  # use your own seed (or none) for a different simulation
m = 10^6;  n = 12;  df = 11;  c = qf(.99, df, df)
v1 = replicate(m, var(rnorm(n,20,2)))
v2 = replicate(m, var(rnorm(n,25,1)))
r = v1/v2;  mean(r > c)
## 0.430304

The histogram below shows all but a few of the $m$ simulated values of $R$ (with $\psi = 4$). [The maximum variance ratio was above 100.] The density curve of $\mathsf{F}(11,11)$ is shown; it runs off the top of the graph. The critical value is indicated by a vertical red line.

enter image description here


Addendum: Minitab 17 has a number of procedures for making power curves, and one of them is for this two-sample test. It looks at the ratio of standard deviations. Here is relevant printout and a power curve from Minitab.

Power and Sample Size 

Test for Two Standard Deviations

Testing (StDev 1 / StDev 2) = 1 (versus >)
Calculating power for (StDev 1 / StDev 2) = ratio
α = 0.01
Method:  F Test


       Sample
Ratio    Size     Power
    2      12  0.429634

enter image description here

Related Question