Can someone tell me why this is wrong:
I want to estimate the goodness-of-fit for a linear mixed effects model. I regress the fitted values against the raw values and calculate an $R^2$.
I suppose an accepted alternative would be to run the model with just the random effect and no fixed effects and compare AIC scores, but I'm specifically interested in the scenario above.
Best Answer
Well $R^2$ is a measure of goodness of fit for any linear model. It is the sum of squares due to the fitted model divided by the total sum of squares (denominator just adds the sum of squares due to the error term). But it is usually not used to assess the quality of the model because as in your scenario since all the dat is used to fit you could be overfitting the data. $R^2$ cannot determine overfitting. Add a variable that is just noise and the $R^2$ will still increase. Criteria like AIC and BIC are better at assessing the model because they penalize the likelihood function for having too many parameters while $R^2$ does not. Adjusted $R^2$ tries to remedy this. But I think it is not used because these other criteria tand to work better.