Solved – Structural equation modelling: model selection

aicmodel selectionstructural-equation-modeling

I am currently trying to fit a structural equation model in R with the Lavaan package. I have this model that fits my data pretty good. This model is what I consider the full model, it has all paths that theoretically make sense. The CFI, RMSEA, SRMR are all within an acceptable range and the X2 is not significant so it seems that the model fits my data well.

There are however, some paths that are not significant. If i leave these non-significant paths out my model fits the data even better.

My problem is now that I am a little bit confused about model selection. Most things that I read about this are a bit vague. Some say that you should modify the full model as less as possible. As my full model is a good fit I could keep it as it is. Others say that I should remove the non-significant paths and compare the models with a likelihood ratio test, AIC, AICc or BIC scores. If I remove the non-significant paths from my model the estimates don't change very drastically but the AICc ends up 15 points lower which would mean that the reduced model is better.

If I would use the AICc scores to select my model shouldn't I than compare all possible combinations of models and select the best or even calculate an averaged model?

The question is, how should I select my final model (unchanged full model, AICc,…)?

Best Answer

(I could not comment yet so I post as an answer...)

I read many paper that used SEM, here is what I have observed from the papers:

  1. Parsimony is better. If you are using SEM as EFA, usually people remove non-sig. path to keep the model simpler.

  2. But if your model contain something that in your field have always been found to be sig. and your model, due to other variables, suggested otherwise. They will report the model with the non-sig. path, and then explore what the supposed sig. path is insignificant. Then usually they will run another SEM without that insig. path and compare the fit indices again.

  3. And yes not just the AICs or BICS, even the fit indices (RMSEA, CFI or chi-sq) very often contradict each other, some authors would report only those support what they did (which I dislike), others would compare everything and explain their decision, either it is more 'theory driven' or 'parsimony'.

Related Question