Solved – Mixed (type III) model ANOVA in R and GraphPad Prism

anovar

Consider the following setup: two mouse strains ("KO" and "WT") have been compared in three independent experiments ("E1", "E2" and "E3"). In each experiment, there were two groups corresponding to the two mouse strains compared, and each group consisted of four different mice. There were all in all 8 mice per experiment, and three experiments (total of 4 x 2 x 3 = 24 mice).

The question is, of course, whether the strains differ.

In R, I would do this as follows, given the variables strain and experiment:

a <- data.frame(
  response= c(100, 110, 120, 130, 200, 210, 220, 230, 105, 115, 125, 135, 205, 
              215, 225, 235, 105, 115, 125, 135, 215, 225, 235, 245),
  experiment= rep( c( "E1", "E2", "E3" ), each= 8 ),
  strain= rep( rep( c( "WT", "KO" ), each= 4 ), 3 ) )

myaov <- aov( response ~ strain + Error( experiment ), data= a ) 
summary( myaov )

However, I have been asked this question by persons who work on a regular basis with GraphPad Prism. It does feature "repeated measures ANOVA" which is (in the help file) said to be equivalent to random block design, but I find the explanations somewhat confusing:

"Repeated measures" vs. "randomized block" experiments

The term repeated measures is appropriate when you made repeated measurements from each subject.

Some experiments involve matching but not repeated measurements. The term randomized-block describes these kinds of experiments. For example, imagine that the three rows were three different cell lines. All the Y1 data came from one experiment, and all the Y2 data came from another experiment performed a month later. The value at row 1, column A, Y1 (23) and the value at row 1, column B, Y1 (28) came from the same experiment (same cell passage, same reagents). The matching is by row.

Randomized block data are analyzed identically to repeated-measures data. Prism always uses the term repeated measures, so you should choose repeated measures analyses when your experiment follows a randomized block design.

My question: how can I analyse these data using GraphPad Prism? Or, alternatively: does the quoted description fit my case?

Best Answer

Repeated measures anova is an old technique that assumes the correlation between any two time points are the same (compound symmetric covariance structure). It also requires a correction to be applied to get correct P-values that account for non-independence of repeated observations within subject. Other approaches work better such as the full likelihood methods of mixed effect models and generalized least squares. R provides many approaches to modeling repeated/longitudinal data and to using realistic correlation structures such as exponential declines over longer time spans. Full likelihood methods are also more robust to missing values that are not missing completely at random. My course notes at http://biostat.mc.vanderbilt.edu/rms have a case study worked out using generalized least squares.

Type I SS:

SS(A)
SS(B|A)
SS(A*B|A,B)

Type II SS:

SS(A|B)
SS(B|A)
SS(A*B|A,B)

Type III SS:

SS(A|B,A*B)
SS(B|A,A*B)
SS(A*B|A,B)

Notice how these different possibilities work. Only type I SS actually uses those SS in the overlapping portion between the circles in the MasterCard symbol. That is, the SS that could be attributed to either A or B, are actually attributed to one of them when you use type I SS (specifically, the one you entered into the model first). In both of the other approaches, the overlapping SS are not used at all. Thus, type I SS gives to A all the SS attributable to A (including those that could also have been attributed elsewhere), then gives to B all of the remaining SS that are attributable to B, then gives to the A*B interaction all of the remaining SS that are attributable to A*B, and leaves the left-overs that couldn't be attributed to anything to the error term.

Type III SS only gives A those SS that are uniquely attributable to A, likewise it only gives to B and the interaction those SS that are uniquely attributable to them. The error term only gets those SS that couldn't be attributed to any of the factors. Thus, those 'ambiguous' SS that could be attributed to 2 or more possibilities are not used. If you sum the type III SS in an ANOVA table, you will notice that they do not equal the total SS. In other words, this analysis must be wrong, but errs in a kind of epistemically conservative way. Many statisticians find this approach egregious, however government funding agencies (I believe the FDA) requires their use.

The type II approach is intended to capture what might be worthwhile about the idea behind type III, but mitigate against its excesses. Specifically, it only adjusts the SS for A and B for each other, not the interaction. However, in practice type II SS is essentially never used. You would need to know about all of this and be savvy enough with your software to get these estimates, and the analysts who are typically think this is bunk.

There are more types of SS (I believe IV and V). They were suggested in the late 60's to deal with certain situations, but it was later shown that they do not do what was thought. Thus, at this point they are just a historical footnote.

As for what questions these are answering, you basically have that right already in your question:

Estimates using type I SS tell you how much of the variability in Y can be explained by A, how much of the residual variability can be explained by B, how much of the remaining residual variability can be explained by the interaction, and so on, in order.
Estimates based on type III SS tell you how much of the residual variability in Y can be accounted for by A after having accounted for everything else, and how much of the residual variability in Y can be accounted for by B after having accounted for everything else as well, and so on. (Note that both go both first and last simultaneously; if this makes sense to you, and accurately reflects your research question, then use type III SS.)

Mixed Model – Numerator DF for Type II and III ANOVA Without Fixed Intercept

For what it is worth, emmeans agrees with SAS on this. The Type III tests are:

> library(emmeans)
> jt3 = joint_tests(results9, mode = "satterth")
> jt3
 model term df1 df2 F.ratio p.value
 a            2   8   5.426  0.0324
 b            1  12  25.852  0.0003
 a:b          2  12   5.359  0.0217

Using a new addition not yet present in the CRAN version, we can see the associated estimable functions:

> attr(jt3, "est.fcns")
$a
     a1 a2 a3 b2 a2:b2 a3:b2
[1,] -1  1  0  0   0.5   0.0
[2,]  0 -1  1  0  -0.5   0.5

$`a:b`
     a1 a2 a3 b2 a2:b2 a3:b2
[1,]  0  0  0  0     1     0
[2,]  0  0  0  0    -1     1

$b
     a1 a2 a3 b2     a2:b2     a3:b2
[1,]  0  0  0  1 0.3333333 0.3333333

These are exactly what you would expect.

The type II tests (via jt2 = joint_tests(results9, mode = "satterth", submodel = "type2")) are identical to the type III ones here, due to the balance.

One interesting thing to me is that the Keward-Roger method yields fairly different (and more conservative) results. This is due to the adjustment that K-R applies to the covariance matrix.

> joint_tests(results9, mode = "kenward")
 model term df1   df2 F.ratio p.value
 a            2 10.67   4.070  0.0486
 b            1 16.00  19.389  0.0004
 a:b          2 16.00   4.019  0.0385

One explanation I can think of is that SAS and joint_tests() both use joint tests of estimable linear functions of the regression coefficients; hence they both explicitly address the question of estimating contrasts related to each effect. However, the appendices of the lmerTest article in JSS (linked in the OP) refer to manipulations of the columns of the design matrix. The way that R constructs the design matrix for this model, there are 3 columns associated with a and therefore the intercept effect becomes part of the a effect in its calculations. In other words, there is an implicit assumption that design-matrix columns associated with predictors are all associated with contrasts; and that isn't the case in the model without intercept.

Best Answer

Related Solutions

ANOVA and MANOVA – How to Interpret Type I, Type II, and Type III ANOVA and MANOVA?

Type I SS:

Type II SS:

Type III SS:

Mixed Model – Numerator DF for Type II and III ANOVA Without Fixed Intercept

Related Question