Solved – Whether to use multivariate F test, df adjusted estimate, or sphericity assumed in repeated measures ANOVA

anovamultivariate analysisrepeated measuressphericity

I have data that involves 2 groups (equal sample size in each) and data for each group over 3 time points (they are actually 3 different monetary reward conditions). I want to investigate within group differences. All time/condition points are important, I don't have a "control" time point.

Any opinions on the following:

  1. If I am comfortable that sphericity is assumed by Mauchly's Test not being violated, would a multivariate Lambda F test statistic be more appropriate or a sphericity assumed estimate (within-condition estimate from SPSS)?

  2. If I choose to go with the understanding that despite a test result saying it is OK, assuming sphericity may still be over-confident, any thoughts on potentially reporting ALL
    Greenhouse-Geiser estimates regardless of Mauchly's or Lambda? I have read that this may reduce the chance of a Type-1 error without having to assume sphericity or equality of covariance matrices. Perhaps too overcautious? Or, is potentially adjusting df's more invasive than assuming sphericity?

Best Answer

so... here is a bit of a dog's breakfast of suggestions

There are more ways to approach this than the options you give yourself. One of them might be to take your three reward levels, one being neutral, and turn them into two reward effects. So, if C is neutral reward, and A and B are test levels makes up an A effect (A-C) and a B effect (B-C) and then compare them to each other. Because there are only two levels sphericity is not an issue. And then you're actually comparing your two effects. Do not make the mistake of testing A-C, and finding it significant and B-C, and finding it not, and then concluding there's a difference between A-C, and B-C. The difference between those two may not be significant in itself.

Mauchly's test, like all such tests, isn't terribly useful. It will fail all of the time with enought power, even if the sphericity violation isn't too bad and you can pass it all of the time with very low power. It definitely shouldn't be used like a hypothesis test in the Neyman-Pearson sense. No test of assumptions should be. Meeting your assumptions is a qualitative decision and Mauchly can help you with that but it's not used as a hard decision rule. Along those same lines, always using GG corrections can reduce the amount of Type I error that you make, as you inquired. However, it can also increase the amount of Type II.

And assuming sphericity isn't invasive at all... Describing it that way makes it sounds like you have a little bit too much reverence for your data and believe there's some church of decisions here. If you want to be conservative in your tests, GG everything and Bonferroni correct. But, if you do that, recognize that you're possibly making Type II errors and note that in your write up. If you don't want to do any of that then don't, but make sure that then you draw weaker conclusions about your tests, especially multiple ones and use them to point the way for future researchers to look.

If you want to go multivariate knock yourself out. It helps with the sphericity issue, if there is one. It doesn't fix multiple testing issues. But you should pick one beforehand and stick with it, not run all kinds of different analyses and see which makes your results look better. That's a whole different level of multiple comparisons. Posting your actual Mauchly test numbers and GG corrections on here might result in you getting some expert advice on how large a violation they are. It's unlikely they're big given that you only have 3-levels.

Speaking of 3-levels, there are no GG corrections for when you have two levels. There is no test of sphericity then. If you decide to make three comparisons, A-B, A-C, B-C, then none of these if GG corrected.

A final option you haven't mentioned is to calculate confidence intervals for each of your 3 comparisons and speak about them rationally. You could alpha adjust them if you wish, or even put two levels of bars on a graph. Then you just describe what are likely and unlikely to be the true values of the effect. So, if A-C does not cross 0 then 0 is an unlikely value. Only the values within the CI are likely.

As you can see, the reason you haven't gotten hard answers to your questions is that, despite your having formulated them well, there are only hard answers to small aspects of what you want to know. That can get firmer when someone has real numbers and hypotheses to deal with. To get more specific help in your description, multiple comparison issues, and some expert advice on how to treat your data, then make a new query with your analyses posted, numbers and what they mean, what your hypotheses are, and what you hope to find out or discuss. That will be more likely to land you some hard advice that's useful.