I have two sets of models. The first set contains a number of logistic models, fitted using glm()
, with different binary dependent variables. The second set contains a number of linear models, fitted with lm()
, with different continuous dependent variables.
All models are testing for differences between participants in one of four conditions (three treatments and one control) coded as a factor (with the control as the contrast), while controlling for a number of demographic variables, some of them factors and some of them continuous.
The sample size is roughly 800 per condition (the conditions aren't perfectly balanced), so about 3,200 in total.
In addition to comparing the treatment conditions to the control through summary()
, however, I would also like to compare the treatment conditions to one another, to see if some are significantly more effective than others. My question is whether to use multcomp()
or emmeans()
to do this.
As I understand it (e.g., from here), the main difference is that emmeans()
uses a t statistic (assuming that hasn’t changed from lsmeans()
) while multcomp()
uses a z statistic, and that the latter therefore tends to result in inappropriately small p values and short confidence intervals.
That would seem to recommend emmeans()
. Or are there ever other considerations making multcomp()
more appropriate?
Best Answer
For
glm
models, both use a z statistic. In general, there is little difference between usingemmeans::contrast()
andmultcomp::glht()
except for user interface. The latter is somewhat harder to use with multi-factor models because there isn't a nice interface for specifying pairwise comparisons of limited groups or marginal averages; but on the other hand, you can specify comparisons inglht()
in the same way as inemmeans()
by usingemm()
instead ofmcp()
. See? emmeans::emm
. Alternatively, you may convert anemmGrid
object to summarize in multcomp via the functionas.glht()
.The default multiplicity adjustment in
glht()
is the single-step method, and that same method is available asadjust = "mvt"
incontrast()
. One situation where you may preferglht()
is if you want to use one of the multi-step adjustment methods it offers, which are not available incontrast()
. (The exception is that those inp.adjust.methods
are available in both.)