Solved – SEM: Robustness of fit indices (CFI, TLI and RMSEA) when data are non-normal

goodness of fitmplusstructural-equation-modeling

Because my data is not normally distributed, I used the Maximum Likelihood Robust Estimator
(MLR) in Mplus. Now I know that this estimator produces a robust chi-square-statistic and robust standard errors, but does it also produce robust fit indices (CFI, TLI & RMSEA)?
Or does Mplus automatically use corrections to adjust for non-normality when calculating these indices?

Best Answer

This is one of a class of more general questions in SEM about how fit indices are calculated - it's not just relevant to Mplus.

The incremental fit indices (CFI, etc) all work by comparing the fitted model chi-square with the null model chi-square. They are not hard to work out. For example:

$$ CFI = (\chi^2_0 - \chi^2_m) / \chi^2_0 $$

TLI/NNFI includes some degree of freedom calculations.

The issue is that there is not consensus on how the null model should be estimated. AMOS and EQS, for example, fix correlations between exogenous measured variables for the null model to zero. Mplus does not. LISREL uses a different chi-square to calculate the null model than Mplus does.

If you fit a parallel factor model with equality constraints on loadings and errors, your null model $\chi^2$ can be better than your fitted model $\chi^2$ - but the null model is supposed to be the worst model that there is.

In short, if you have any doubt, don't trust your incremental fit indices - work them out. There's a paper on this: Widaman, K. F., & Thompson, J. S. (2003). On specifying the null model for incremental fit indices in structural equation modelling. Psychological Methods, 8(1), 16-37.

RMSEA is based only on chi-square. If you're worried, work it out.

$RMSEA = \sqrt((\chi^2_m - df)/(df(N-1))$

However, if when you shift from regular ML to a robust estimator, if RMSEA changes, you can be pretty sure that it's using the robust estimator.

One interesting place that Mplus does NOT change is in the calculation of the relative fit indices - such as AIC. Mplus does not use the robust $\chi^2$ in the calculation of AIC, it uses the ML version. I'm not sure about other programs, but that's not the behavior I'd have expected. (Because it uses the log likelihood, rather than the $\chi^2$, to calculate AIC).