I'm working on a 2-way ANOVA analysis with a 2×2 design – two main effects with 2 levels each. In running the analysis, I obviously get the interaction as well as each main effect.
Are the results (p and F values) for the main effects the same as if I had simply run a standard ANOVA on each of the main effects independently? That is my interpretation, but when I actually do the test, I get different p and F values.
Best Answer
In a balanced two-way design (all cells equally frequent), the two-way ANOVA without interaction will basically return the same p values for the main effects as the two one-way ANOVAs. (Slightly different degrees of freedom but the hypotheses are the same.)
In the model with interaction, the same is true only if you use orthogonal contrasts (e.g. effect coding aka sum contrasts, or Helmert contrasts). If you use treatment contrasts (aka dummy coding), then your "main" effects have a completely different meaning than in the one-way ANOVAs (or the two-way ANOVA without interaction) and thus their tests are usually meaningless.
In short: In the model with interaction, just look at the interaction term. If you are interested in main effects, then run the model without interaction or use proper contrasts.