I have run two psychological experiments. The dependent variable is a rating provided by the participant, that is, an integer number from 0 to 100.
The first experiment is a mixed 2×2 design, with one between-subject factor (treatment) and one within-subject factor (question). Each of these factors has two levels. That is, participants are asked two questions in the experiment (Q1 and Q2), while a factor varies systematically between the two groups (Treatment1 and Treatment2).
The second experiment is identical, except for that the design becomes completely within-subjects. That is, it is a repeated measures 2×2 design; participants are asked four questions that encode the two manipulations in a factorial form: Q1Treatment1, Q1Treatment2, Q2Treatment1, and Q2Treatment2.
I am interested in the potential interactions between the two factors (question and treatment).
Normally, I would conduct repeated measures ANOVAs for these experiments. However, the problem is that my data deviate strikingly from normality (in fact, so do the residuals). Therefore, I am looking for nonparametric equivalents to ANOVA for these two designs.
I can use both SPSS and R.
Best Answer
In a 2 x 2 design, it is fairly easy to run a bootstrap test of the interaction. Let define the four conditions as a, b, c and d. The conditions $a$ and $b$ are the question factor for the treatment and $c$ and $d$ are the question factor for the control condition. The mean interaction contrast (MIC) is given by
$ (a + d) - (b + c).$
It quantifies the amount of non-additivity in the dataset. If MIC is zero, it means that there is a main effect of questions (there is an increase--or decrease-- from Q1 to Q2), a main effect of conditions (there is an increase --or decrease-- from control to treatment) and no interaction. If such is the case, mean in $b$ is a few points above mean in $a$, and mean in $c$ is also the same amount of points above the mean in $d$. As of treatment, the same occur (treatment measures are a few point above control measures). Defining the first increment as $d_1$ and the second as $d_2$, the means are thus
$ \left(\begin{matrix}M_a \;\;\; M_b \\ M_c\;\;\;M_d \end{matrix}\right) = \left(\begin{matrix}M_a \;\;\;M_a+d_1 \\ M_c \;\;\; M_c+d_1 \end{matrix}\right) = \left(\begin{matrix}M_a \;\;\; M_a+d_1 \\ M_a+d_2\;\;\;M_a+d_2+d1 \end{matrix}\right) $
so that
$MIC = (M_a+(M_a+d_1+d_1)) - ((M_a+d_1)+(M_a+d_2)) = 0$.
Thus, to do a boostrap estimate, sub-samples in the groups with replacement, and compute MIC. Repeat this a very large number of times (say 5,000). Finally, find the range in which 95% of the MIC found are located. If this interval includes 0, then the interaction is not significantly different from zero.
This reasoning works for a fully between group design. In a mixed design, you have to select pairs of scores randomly before computing MIC (preserving subjects' two measures).