I have two multilevel path models, one is nested in the other and I want to compare the models to see whether I should prefer the full or nested model.
The models have cross-level interactions and I am currently using Mplus with a bayesian estimator. However, Mplus doesn't have (as far as I have been able to find) a way to compare models outside of things the DIC (for example DIFTEST doesn't work for Bayesian ML-SEM).
Similarly, R's Mplusautomation package has a model comparison function, but it doesn't seem to produce something like a chi-square Log Likelihood ratio for Bayesian estimators. Although alternative packages like Lavaan handle ML-SEM and can be compared relatively easily with a Likelihood Ratio Test, it doesn't currently support cross-level interactions (as far as I know).
I am looking for suggestions on how to compare two nested multilevel path models with cross level interactions. I am aware you can use the DICs, but I am a bit lost as to what constitutes a meaningful difference between two DICs. So any advice in this direction would also be helpful.
Best Answer
Variability of casewise likelihoods forms a basis for calculating a SE or CI around (differences in) information criteria, but I think that is also not available in Mplus.
blavaanobtains them from theloopackage, butblavaandoes not yet handle MLSEM, although you could simply code up the model manually in Stan (rstanpackage) to utilize theloopackage.If the models are parametrically nested (i.e., the more restricted model is equivalent to the less restricted model by fixing one or more of its parameters), then you can conduct a Bayesian version of the Wald test from Mplus version 8.4: http://www.statmodel.com/download/BayesFit.pdf
Note that Wald (and score) tests are asymptotically equivalent to a LRT (Buse, 1982), so hopefully this provides the information you are looking for.