Or is that not necessarily a reasonable approach?
For example, my model is a linear mixed-effects model with only one explanatory variable, such that
model <- lme(log(continuousvariable + 0.1) ~ factor(2-levels), random=~1 | block/plot, data=stand)
running ANOVA on the model yields a p-value < 0.05 for the factor.
Usually I follow-up with emmeans (used to do lsmeans back in the day) to get an idea of the values by factor, Tukey's p-value adjustment, and cld (I know there's some hand-waving here) but in this case there's only 2-levels being compared. Doing some research on Russ Lenth's FAQ for emmeans, p-value adjustments can't be made where there's only two means being compared. I still get results running emmeans, but it's not initially clear to me what test is actually being done here…i.e., clearly there's no p-value adjustment. For example… output for the following code
em <- emmeans(model, ~ factor(2-levels))
multcomp::cld(em, Letters=c("abcdefghi"), type="response") #back-transformation of emmeans
#output is something along the lines of
pm2 response SE df lower.CL upper.CL .group
f1 471 149 3 172 1293 a
f2 2765 773 3 1135 6734 b
Degrees-of-freedom method: containment
Confidence level used: 0.95
Intervals are back-transformed from the log(mu + 0.1) scale
Tests are performed on the log scale
significance level used: alpha = 0.05
NOTE: Compact letter displays can be misleading
because they show NON-findings rather than findings.
Consider using 'pairs()', 'pwpp()', or 'pwpm()' instead
However… I'm not clear what test was done exactly…? Any thoughts and-or suggestions? Maybe I just didn't dig deep enough or this isn't a logical approach.
Best Answer
It is running the same test you get with
pairs(em)
, i.e. a t test on the difference between the means, with standard error derived from the model structure and no P-value adjustment since there is only one test. If the P value is less than .05, you will get two different letters (which is the case shown); and if ghe P value is greater than .05 you will get one letter because they'll be in the same grouping.I do not recommend CLDs, as you can see in the annotations. Perhaps the confusion it causes with just two means is yet another argument against them.