ANOVA – Post-Hoc for 2×2 Mixed Design ANOVA Using SPSS

anovapost-hocspss

I am analysing an experiment run with 20 participants using a $2\times 2$ mixed design ANOVA. The experiment has

one within subjects variable A with two levels (a1, a2)
one between subjects variable B with two levels (b1, b2)

The following are the results of the ANOVA analysis:

B F(1,18) = 5,84 p<0,026493 SS=1,44 MSe=0,25

A F(1,18) = 11,1 p<0,003734 SS=1,25 MSe=0,11

B*A F(1,18) = 7,16 p<0,015412 SS=0,81 MSe=0,11

Is there any reason to run post-hoc test in this context? SPSS complains about variables having less than 3 levels and, for this reason, doesn't allow me to run Post-hoc tests.

Question
Is it possible to run bonferroni post-hoc test in this kind of scenario? If not, why? If yes, how to do it in SPSS?

Best Answer

There's no use for a post hoc test here. What could you possibly find in a post hoc that isn't obvious from the ANOVA? There's a main effect of A and a main effect of B and an interaction.

Some people do them to do something like test whether A1B1 - A1B2 is significant while A2B1 - A2B2 is not. They find that result and report it as important but it's meaningless because it tells you less than the interaction already told you. Significant and not-significant is not a test of differences between conditions dependent upon the level of the other condition. The interaction already was that test.

Related Solutions

Solved – Post hoc test in a 2×3 mixed design ANOVA using SPSS

Answer edited to implement encouraging and constructive comment by @Ferdi

I would like to:

provide an answer with a full contained script
mention one can also test more general custom contrasts using the /TEST command
argue this is necessary in some cases (ie the EMMEANS COMPARE combination is not enough)

I assume to have a database with columns: depV, Group, F1, F2. I implement a 2x2x2 mixed design ANOVA where depV is the dependent variable, F1 and F2 are within subject factors and Group is a between subject factor. I further assume the F test has revealed that the interaction Group*F2 is significant. I therefore need to use post hoc t-tests to understand what drives the interaction.

MIXED depV BY Group F1 F2 
  /FIXED=Group F1 F2 Group*F1 Group*F2 F1*F2 Group*F1*F2 |  SSTYPE(3) 
  /METHOD=REML 
  /RANDOM=INTERCEPT | SUBJECT(Subject) COVTYPE(VC) 
  /EMMEANS=TABLES(Group*F2) COMPARE(Group) ADJ(Bonferroni)
  /TEST(0) = 'depV(F2=1)-depV(F2=0) differs between groups' 
    Group*F2 1/4 -1/4 -1/4 1/4 
    Group*F1*F2 1/8 -1/8 1/8 -1/8 -1/8 1/8 -1/8 1/8 
  /TEST(0) = 'depV(Group1, F2=1)-depV(Group2, F2=1)' Group 1 -1
    Group*F1 1/2 1/2 -1/2 -1/2 
    Group*F2 1 0 -1 0  
    Group*F1*F2 1/2 0 1/2 0 -1/2 0 -1/2 0 .

In particular the second t-test corresponds to the one performed by the EMMEANS command. The EMMEANS comparison could reveal for example that depV was bigger in Group 1 on the condition F2=1.

However the interaction could also be driven by something else, which is verified by the first test: the difference depV(F2=1)-depV(F2=0) differs between groups, and this is a contrast you cannot verify with the EMMEANS command (at least I did not find an easy way).

Now, in models with many factors it is a bit tricky to write down the /TEST line, the sequence of 1/2, 1/4 etc, called L matrix. Typically if you get the error message: "the L matrix is not estimable", you are forgetting some elements. One link that explains the receipt is this one: https://stats.idre.ucla.edu/spss/faq/how-can-i-test-contrasts-and-interaction-contrasts-in-a-mixed-model/

Solved – Mixed ANOVA post hoc analysis (and multiple comparisons)

The effect you're looking for is the Class:time interaction. That 2x2 interaction tells you that the effect of B-A differs across time. That's the critical thing to measure because it takes into account potential baseline variability in groups A and B at the first time. Given that it's the effect, B-A, across time that's being tested then there is no, "appears in time 2" etc. because it's very existence is measured in comparison to time 1.

Consider a situation where at time 1 group A tends to score higher than B. Then at time 2 group B scores higher. Each of those individual differences might not even be very large but the difference between the differences is going to be bigger and reveal what the effect of receiving the drug over time is. If you carry out that thought experiment a little more you realize that looking at the simple effects is a fools errand.

Going spelunking with post hoc tests with a 2x2 always ends up in logical fallacies of comparing significant v. not significant. Don't do it. Report your interaction and describe the pattern in the data.

Best Answer

Related Solutions

Solved – Post hoc test in a 2×3 mixed design ANOVA using SPSS

Solved – Mixed ANOVA post hoc analysis (and multiple comparisons)

Related Question