Two-Way ANOVA – Handling Unbalanced and Nested TYPE III with Significant Results

anovaexperiment-designr

I have an unbalanced design with nested factors. In particular, the nested factor is obesity, with levels: of obese and nonobese. The other factor is group: women with syndrome, control women, and control men. The response let's say is a metabolite.

What I have done, is to set a linear model, with the following contrasts, due to the nature of the problem: the coefficients of the variates sum up to zero.

Then, in R one can achieve this with:

mdl <- lm(Y ~ obesity*group, 
    contrasts=list(obesity=contr. sum, 
    group=contr.sum)
car::Anova(mdl, type=3)

Then the output is

    Anova Table (Type III tests)

    Response: Y
                   Sum Sq Df   F value  Pr(>F)    
    (Intercept)    549.02  1 1117.2302 < 2e-16 ***
    obesity         0.19  1    0.3927 0.53446    
    group            2.09  2    2.1311 0.13199    
    obesity:group   3.21  2    3.2632 0.04866 *  
    Residuals       19.66 40                      
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

I tested for the assumptions of normality of the residuals and homogeneity, the model accomplishes this.

Now, as you can see, the interaction is significant, however, running TukeyHSD test, is not well suited because, from the help in R, it says:

" This is a generic function: the description here applies to the method for fits of class "aov" "

Thus, aov functions are the wrapper of anova(lm), but it does not take into consideration the contrasts, and it only does the sequential sum of squares

So I think, that the problem could be resolved with "marginal means"

The problem: I have no idea how to insert the formula with the error term.

I tested:

mdl2  <- aov_car(Y ~ obesity*group  + 
                 Error(patients)

Where patients are the subjects. I know that the error term, is the random mixed effect, (I think..)

Then I can run

emmeans(mdl2)

with the same output,

But I cannot figure out which is the significant group of interaction that the above results say.

So the question is why is significant the ANOVA but not the interaction ? Should I run pairwise t test with interaction ? I mean, interaction(obesity,group) and then run pairwise t.test ?

I have seen several posts about this issue, in particular, I think that the question is already answered on this package: https://cran.r-project.org/web/packages/emmeans/index.html

More than the data itself, it's a question of statistics

Best Answer

Whether you insist on keeping "obesity" as an all-or-none marker or, as Frank Harrell rightly suggests, you treat it as one or more continuous predictors (e.g., including both height and weight in some way), you have several potential sources of confusion in the way you are approaching this problem.

First, as the manual page for car::Anova says:

"Be careful of type-III tests... type-II tests are invariant with respect to (full-rank) contrast coding. If you don't understand this issue, then you probably shouldn't use Anova for type-III tests."

It's not clear how useful type-III tests will be, particularly with an unbalanced design and a potentially significant interaction. Your choice of type-III tests also seems to be forcing you to use sum contrasts. (I personally find the coefficients from sum contrasts harder to think about than those from the default treatment contrasts, although that might be my limitation and you might find otherwise.)

Second, it's not clear that you have more than one observation per participant for any one response variable. If you don't, then there's no "random effect" to deal with in univariate analysis, as you seem to be trying to do in your code. (If you have multiple response variables for each participant you would benefit from some type of multivariate analysis.)

Third, you ask "why is significant the ANOVA but not the interaction?" This might be a problem with terminology, but that's not what you show: the interaction term is significant (if barely) by the traditional p < 0.05 criterion.

Perhaps you are having problems identifying a particular "significant" pairwise comparison among the combinations of groups despite that overall finding of a "significant" interaction. That can happen in some circumstances, particularly when the overall "significance" is borderline as in the case you display.

Doing a set of pairwise t-tests as you propose would be an error unless you took into account the multiple-comparisons problem in some way. The TukeyHSD is only one way to do that; there are others.

Type I SS:

SS(A)
SS(B|A)
SS(A*B|A,B)

Type II SS:

SS(A|B)
SS(B|A)
SS(A*B|A,B)

Type III SS:

SS(A|B,A*B)
SS(B|A,A*B)
SS(A*B|A,B)

Notice how these different possibilities work. Only type I SS actually uses those SS in the overlapping portion between the circles in the MasterCard symbol. That is, the SS that could be attributed to either A or B, are actually attributed to one of them when you use type I SS (specifically, the one you entered into the model first). In both of the other approaches, the overlapping SS are not used at all. Thus, type I SS gives to A all the SS attributable to A (including those that could also have been attributed elsewhere), then gives to B all of the remaining SS that are attributable to B, then gives to the A*B interaction all of the remaining SS that are attributable to A*B, and leaves the left-overs that couldn't be attributed to anything to the error term.

Type III SS only gives A those SS that are uniquely attributable to A, likewise it only gives to B and the interaction those SS that are uniquely attributable to them. The error term only gets those SS that couldn't be attributed to any of the factors. Thus, those 'ambiguous' SS that could be attributed to 2 or more possibilities are not used. If you sum the type III SS in an ANOVA table, you will notice that they do not equal the total SS. In other words, this analysis must be wrong, but errs in a kind of epistemically conservative way. Many statisticians find this approach egregious, however government funding agencies (I believe the FDA) requires their use.

The type II approach is intended to capture what might be worthwhile about the idea behind type III, but mitigate against its excesses. Specifically, it only adjusts the SS for A and B for each other, not the interaction. However, in practice type II SS is essentially never used. You would need to know about all of this and be savvy enough with your software to get these estimates, and the analysts who are typically think this is bunk.

There are more types of SS (I believe IV and V). They were suggested in the late 60's to deal with certain situations, but it was later shown that they do not do what was thought. Thus, at this point they are just a historical footnote.

As for what questions these are answering, you basically have that right already in your question:

Estimates using type I SS tell you how much of the variability in Y can be explained by A, how much of the residual variability can be explained by B, how much of the remaining residual variability can be explained by the interaction, and so on, in order.
Estimates based on type III SS tell you how much of the residual variability in Y can be accounted for by A after having accounted for everything else, and how much of the residual variability in Y can be accounted for by B after having accounted for everything else as well, and so on. (Note that both go both first and last simultaneously; if this makes sense to you, and accurately reflects your research question, then use type III SS.)

Solved – Analyse unbalanced repeated measures 2x2x2x2 type II anova interactions

As this is still a very simple design you should stick with the classical analysis, ANOVA.

I recommend (in agreement with e.g., Maxwell & Delaney) to use Type III sums of squares for this problem and inspect the interactions by running separate ANOVAs for the levels for one of the factors involved in the interactions (similar to so called simple main effects analysis). This will essentially tell you what drives the interaction and it is convenient to report (it is generally recommended to plot the interaction before doing so).

Note that you need to use contrast coding before running ANOVAs in R with the 3 sums of squares by calling the following before your ANOVAs:

options(contrasts=c("contr.sum","contr.poly"))

Best Answer

Related Solutions

ANOVA and MANOVA – How to Interpret Type I, Type II, and Type III ANOVA and MANOVA?

Type I SS:

Type II SS:

Type III SS:

Solved – Analyse unbalanced repeated measures 2x2x2x2 type II anova interactions

Related Question