I am using mixed effect models to predict a time series of data. I am using lmertest() in R to overload lmer() to gain p values via Satterthwaite approximation. The general model for each formula in R syntax is:
Dependent Variable ~ time^2 + time + (time | random effect)
For those not versed in R, this is predicting my dependent variable using the fixed effect of time and time squared (to mimic a quadratic function) whilst allowing the second and trailing coefficients in a quadratic function to vary per time series. All models are using maximum likelihood.
My model appears to account for a reasonable amount of variance (~ .06 R^2m, .94 R^2c) but I'm having difficulty understanding the p values.
my intercept is highly non significant (~.76), but both the coefficients return <.001).
My questions are therefore:
What is the Satterthwaite approximation actually doing to create these values?
My fixed effects appear to be highly significant whilst my intercept isn't, how should I interpret this finding? My gut tells me this means that the model could find good coefficients which meant time could predict my DV, but that the intercepts the model found cannot be trusted as assisting with predictions?
Is there a better way to force out p-values from a mixed effect model than this? I'm considering using the anova() function from the car package which does a wald test mainly.
How concerned should I be about the non-significant intercept, given my question is does my nature in general tend to follow a concave polynomial shape over time?
Cheers.
Best Answer
It is not a good idea to be too concerned with p-values in mixed models. They are omitted from
lme4
by the authors for good reasons, and "forcing" (as you put it) p-values out of the model is regarded by many as a very questionable thing to do. Moreover, since you appear to be focused on prediction rather than inference, a better approach may be to use cross-validation. Here I will quote Douglas Bates, the primary author oflme4
, writing on the r-sig-me mailing list some years ago:Anyway, to answer the questions at hand, Satterthwaite's method is a way to approximate the degrees of freedom that Douglas Bates was describing above.
As for the non-significant fixed intercept, one way to interpret this is that, at some arbitrary level of significance (perhaps 5% if you follow the convention in many fields), the intercept may in fact be zero. Perhaps if you had a larger sample, it would be different from zero (one reason for not relying heavily on p-values in general, not just in mixed models). In other words, perhaps the actual data generating process that you are modelling results in an expected value of zero when other covariates are also zero (and that scenario may be total nonsense in this particular study, or it may be fine). A plot of the data may be very revealing regarding this.
I would also question whether it is a good idea to fit random slopes for the linear term but not the quadratic term. By doing so, you are allowing each group to have it's own linear term, yet the overall shape is constrained to be the same, so you are allowing a a shift in each parabola, but fixing the shape. Is this indicated by the relevant theory of whatever data generating process you are modelling ?