Answer edited to implement encouraging and constructive comment by @Ferdi
I would like to:
- provide an answer with a full contained script
- mention one can also test more general custom contrasts using the /TEST command
- argue this is necessary in some cases (ie the EMMEANS COMPARE combination is not enough)
I assume to have a database with columns: depV, Group, F1, F2. I implement a 2x2x2 mixed design ANOVA where depV is the dependent variable, F1 and F2 are within subject factors and Group is a between subject factor. I further assume the F test has revealed that the interaction Group*F2 is significant. I therefore need to use post hoc t-tests to understand what drives the interaction.
MIXED depV BY Group F1 F2
/FIXED=Group F1 F2 Group*F1 Group*F2 F1*F2 Group*F1*F2 | SSTYPE(3)
/METHOD=REML
/RANDOM=INTERCEPT | SUBJECT(Subject) COVTYPE(VC)
/EMMEANS=TABLES(Group*F2) COMPARE(Group) ADJ(Bonferroni)
/TEST(0) = 'depV(F2=1)-depV(F2=0) differs between groups'
Group*F2 1/4 -1/4 -1/4 1/4
Group*F1*F2 1/8 -1/8 1/8 -1/8 -1/8 1/8 -1/8 1/8
/TEST(0) = 'depV(Group1, F2=1)-depV(Group2, F2=1)' Group 1 -1
Group*F1 1/2 1/2 -1/2 -1/2
Group*F2 1 0 -1 0
Group*F1*F2 1/2 0 1/2 0 -1/2 0 -1/2 0 .
In particular the second t-test corresponds to the one performed by the EMMEANS command. The EMMEANS comparison could reveal for example that depV was bigger in Group 1 on the condition F2=1.
However the interaction could also be driven by something else, which is verified by the first test: the difference depV(F2=1)-depV(F2=0) differs between groups, and this is a contrast you cannot verify with the EMMEANS command (at least I did not find an easy way).
Now, in models with many factors it is a bit tricky to write down the /TEST line, the sequence of 1/2, 1/4 etc, called L matrix. Typically if you get the error message: "the L matrix is not estimable", you are forgetting some elements. One link that explains the receipt is this one: https://stats.idre.ucla.edu/spss/faq/how-can-i-test-contrasts-and-interaction-contrasts-in-a-mixed-model/
In a 2 x 2 design, it is fairly easy to run a bootstrap test of the interaction. Let define the four conditions as a, b, c and d. The conditions $a$ and $b$ are the question factor for the treatment and $c$ and $d$ are the question factor for the control condition. The mean interaction contrast (MIC) is given by
$ (a + d) - (b + c).$
It quantifies the amount of non-additivity in the dataset. If MIC is zero, it means that there is a main effect of questions (there is an increase--or decrease-- from Q1 to Q2), a main effect of conditions (there is an increase --or decrease-- from control to treatment) and no interaction. If such is the case, mean in $b$ is a few points above mean in $a$, and mean in $c$ is also the same amount of points above the mean in $d$. As of treatment, the same occur (treatment measures are a few point above control measures). Defining the first increment as $d_1$ and the second as $d_2$, the means are thus
$
\left(\begin{matrix}M_a \;\;\; M_b \\ M_c\;\;\;M_d \end{matrix}\right) =
\left(\begin{matrix}M_a \;\;\;M_a+d_1 \\ M_c \;\;\; M_c+d_1 \end{matrix}\right) =
\left(\begin{matrix}M_a \;\;\; M_a+d_1 \\ M_a+d_2\;\;\;M_a+d_2+d1 \end{matrix}\right)
$
so that
$MIC = (M_a+(M_a+d_1+d_1)) - ((M_a+d_1)+(M_a+d_2)) = 0$.
Thus, to do a boostrap estimate, sub-samples in the groups with replacement, and compute MIC. Repeat this a very large number of times (say 5,000). Finally, find the range in which 95% of the MIC found are located. If this interval includes 0, then the interaction is not significantly different from zero.
This reasoning works for a fully between group design. In a mixed design, you have to select pairs of scores randomly before computing MIC (preserving subjects' two measures).
Best Answer
If you are not really interested in the comparison between groups go for the 3-way definitely. The 4-way case will end up with A LOT of interactions for you to study. This is very complex and tiring...