Mixed Model – Understanding CLD Test When Running Emmeans for Two Means Comparison

lsmeansmixed model

Or is that not necessarily a reasonable approach?

For example, my model is a linear mixed-effects model with only one explanatory variable, such that

model <- lme(log(continuousvariable + 0.1) ~ factor(2-levels), random=~1 | block/plot, data=stand)

running ANOVA on the model yields a p-value < 0.05 for the factor.

Usually I follow-up with emmeans (used to do lsmeans back in the day) to get an idea of the values by factor, Tukey's p-value adjustment, and cld (I know there's some hand-waving here) but in this case there's only 2-levels being compared. Doing some research on Russ Lenth's FAQ for emmeans, p-value adjustments can't be made where there's only two means being compared. I still get results running emmeans, but it's not initially clear to me what test is actually being done here…i.e., clearly there's no p-value adjustment. For example… output for the following code

em <- emmeans(model, ~ factor(2-levels))
multcomp::cld(em, Letters=c("abcdefghi"), type="response") #back-transformation of emmeans

#output is something along the lines of

pm2 response  SE df lower.CL upper.CL .group
 f1       471 149  3      172     1293  a    
 f2      2765 773  3     1135     6734   b   

Degrees-of-freedom method: containment 
Confidence level used: 0.95 
Intervals are back-transformed from the log(mu + 0.1) scale 
Tests are performed on the log scale 
significance level used: alpha = 0.05 
NOTE: Compact letter displays can be misleading
      because they show NON-findings rather than findings.
      Consider using 'pairs()', 'pwpp()', or 'pwpm()' instead

However… I'm not clear what test was done exactly…? Any thoughts and-or suggestions? Maybe I just didn't dig deep enough or this isn't a logical approach.

Best Answer

It is running the same test you get with pairs(em), i.e. a t test on the difference between the means, with standard error derived from the model structure and no P-value adjustment since there is only one test. If the P value is less than .05, you will get two different letters (which is the case shown); and if ghe P value is greater than .05 you will get one letter because they'll be in the same grouping.

I do not recommend CLDs, as you can see in the annotations. Perhaps the confusion it causes with just two means is yet another argument against them.

Related Question