Mixed Model – Functions in R to Compute Effect Sizes Based on Linear Mixed Effects Models

Question

after researching on how to get estimated effect sizes from linear mixed effects models in R, I still do not understand how this can be properly done. Is there any function in R that does that? I am not lazy but I am working both with lmer and glmer on three types of data (reading times, reaction times, accuracy) and I really do not understand what there is to be done. People say that the estimated fixed effects coefficients are the effect size, or talk about standardization but I do not really understand what that implies really. I also do not really understand whether computing the effect size is different depending on coding scheme of the fixed effects (deviation vs treatment coding). Westfall and colleagues (2014) mention how the effect size could be calculated (the estimated slope coefficient for a given fixed effect / summed variances of all varying intercepts and slopes and residual variance) but I worry that the computing mode depends on the coding scheme, on whether people conduct lmer vs glmer analyses. On the other hand, how should one best report these results? It's all very confusing to me and I would definitely appreciate your help.

References:

Westfall, J., Kenny, D. A. and Judd, C. M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General 143(5): 2020–2045, DOI: https://doi.org/10.1037/xge0000014

Later edit:
There seems to exist a function that computes this, which is implemented in the EMAtools package: lme.dscore. I am nevertheless interested in your input regarding the question that I posted and also with regard to prior experience with this function. Many thanks and happy Monday!

Answer 1

I personally find reports of "effect sizes" to be less compelling than seeing results in original scales. My hesitation to recommend "effect size" measures is enhanced by the extra difficulties in just what an "effect size" means in mixed models or generalized linear models.

The R emmeans package provides a way to calculate Cohen's d from many types of models when it makes sense, with an eff_size() function. See the pairwise comparisons section of the vignette for an introduction.

Nevertheless, the package author, Russ Lenth, discusses why presenting Cohen's d isn't always a good idea, in comments on this answer about the degrees of freedom (d.f) to specify to get an "effect size" for a mixed model. In particular:

In a simple lm() model, things are a lot more straightforward. In a mixed model, there is a lot more ambiguity. But to me, the d.f. to use is the smallest issue. What's bigger is deciding what you are even talking about when you compute an effect size. There is a lot of discussion on this, and from where I sit, the question is nearly unanswerable. I don't really believe in effect sizes. I provided the function because I thought [it] was important to account for uncertainty in the SD estimate if you insist on computing an effect size.

As Robert Long points out, Cohen's d doesn't make sense for generalized linear models. He recommends using odds ratios as a measure of "effect size" for logistic models.

See if you can present your results in the original scales as much as possible. That will remove ambiguity in the meaning of the term "effect size" and be most directly related to the things that you are measuring.