If each subject has repeated a test multiple times, I am able to compute the mean and variance of their performance. (I am assuming the results to be normally distributed).
1) If it is the variance that is of most interest to me then I believe that it is OK to summarise the variance of a group of subjects by taking the mean of their variances?
2) Would it also be meaningful to look at the variance of the subject variances?
3) I believe this (or a similar) statistic will be required if I am to use a T-test to assess whether the difference in variance between groups is statistically significant?
Very simple example (edited to correct the variances as noted in one answer)
Andrew: 5, 2, 7 (var 4.22 6.33)
Bob: 3, 3, 2 (var 0.22 0.33)
Charles: -2, 1 , 0 (var 1.55 2.33)
Diane: 4, 2, 3 (var 0.66 1.00)
Elsa: 6, -4, -1 (var 17.55 26.3)
Fran: 6, 0, 3 (var 6.00 9.00)
Male mean variance (Andrew, Bob and Charles): 2.00 3.00
Female mean variance (Diane, Elsa and Fran): 8.07 12.1
Question: Is there a significant difference between the male and female within-subject variance for this test?
Best Answer
You can test this by fitting a linear mixed model. A linear mixed model is like a multiple regression model but you can have random effects. The random effects part is needed because you have multiple tests per observation. You will then model score as a function of sex and test, and the subjects are your random effects. Let's enter your test data in R:
The data would look like this:
Now we fit two models. Both models use sex and test (1-3 in this case) as fixed effects and subject as random effect. The difference between the models is that in the second model, the variance is allowed to differ between women and men. We then compare the models using the anova() command, and if there is a significant difference, this indicates that the more complex model (the one with the differing variances per sex) provides a better fit, and we thus have indirect evidence that the difference in variance is statistically significant:
There was no difference in this example. But if we change the score for the last female a little (changing score from 3 to -17, increasing the variance) and run the m2 model and the comparison again:
Now we see a difference in AIC and logLik, and a low p-value which indicates a difference in variance between the sexes.