A reviewer is asking me to test for homoscedasticity with "appropriate tests", as opposed to visual inspection of residual plots. I haven't found any such test (I guess he wants a p-value). Does such a test exist and if not, is there something I can cite to justify not presenting a p-value?
lmer Models – Testing for Homogeneity
heteroscedasticitylme4-nlmemixed model
Related Solutions
I think Question 1 and 2 are interconnected. First, the homogeneity of variance assumption comes from here, $\boldsymbol \epsilon \ \sim \ N(\mathbf{0, \sigma^2 I})$. But this assumption can be relaxed to more general variance structures, in which the homogeneity assumption is not necessary. That means it really depends on how the distribution of $\boldsymbol \epsilon$ is assumed.
Second, the conditional residuals are used to check the distribution of (thus any assumptions related to) $\boldsymbol \epsilon$, whereas the marginal residuals can be used to check the total variance structure.
The funny residuals almost certainly relate to the quantization of the response to the set $\left\{9, 10, 11, 12, 13 \right\}$. A normal distribution, conditional on covariates, can't do that--the event has probability zero. As you've said in your update, this quantization is due to rounding (ie, the actual values are not exactly $9.0$ or $13.0$ but were just rounded to those values).
I think this is a bit of a case of "good news/bad news." On the one hand, if the outcome has been rounded, that's really a complicated form of interval censoring, so non-parametric inference won't help. I hypothesize that the point estimates will be at least somewhat biased due to the censoring, and you would need to use a method that models this in order to reduce this bias. FWIW, I don't see any readily available software that does this for mixed models. The R package grouped fits these models for vanilla GLMs.
On the other hand, the rounding may not be a huge issue. In the first residuals vs fitted plot in which you regress Buried there seems there was a tendency to avoid prediction of non-integer values, which makes me worry that the model could be optimistic with its standard error estimates due to the quantization. This doesn't seem to be so true in the second model where EggLaid was the predictor. It might be worth running a simulation study to bound the bias, because I don't have a great intuition about how small or big this bias might be.
In conclusion, I would say yes, you should bootstrap, and that ideally you should use a semi-parametric bootstrap that jitters the response variables with a bit of noise. It looks like this happen already if you use the function bootMer in lme4. From the docs:
If ‘use.u’ is ‘FALSE’ and ‘type’ is ‘"parametric"’, each simulation generates new values of both the “spherical” random effects u and the i.i.d. errors epsilon, using ‘rnorm()’ with parameters corresponding to the fitted model ‘x’.
These iid errors epsilon will mean that the bootstrapped sample is no longer quantized. So as long the the initial residual variance is approximately correct, the bootstrap will do something reasonable.
Best Answer
Yes you could use for example Levene's test using the
leveneTest()function from thecarpackage. Here's an example with theMachinesdataset from thenlmepackage:Since the result is not significant, the assumption of equal variances (homoscedasticity) is met.
Also check
?leveneTestfor more options.Let's compare this with a boxplot of the residuals:
Since the reviewer seems to want a "formal test", it will probably be difficult to convince him accepting your visual inspection, despite, in my opinion, this would be the way to go. Maybe someone else has an actual reference why checking those assumptions visually is superior compared to "formal tests".
Edit to address comment by @D_Williams below:
A good and strongly cited paper by Zuur et al. 2010 may help to convince your reviewer regarding visual inspection of residuals to test for homogeneity of variances. Also here's is the link to the book Mixed Effects Models and Extensions in Ecology with R.