I have a question about model selection when using AIC/BIC. So, if two model structures are totally different, can I still directly apply AIC and BIC?
Also, for a hierarchical model, how to compute the total number of parameters when computing AIC and BIC?
Thanks.
Solved – Prerequisites for AIC/BIC model comparison
aicbicmultilevel-analysis
Best Answer
My understanding of AIC and BIC is that they let you compare the fits of non-nested models. So as long both models are fit to the same data set, a comparison seems justified even if they are "totally different". As for the number of free parameters in a hierarchical model, I think you should use the number of parameters that are being estimated, which means (1) the fixed effects, (2) the random effects variance components, and (3) the covariances of the random effects, if any. This thread provides a more in-depth discussion: https://stat.ethz.ch/pipermail/r-sig-mixed-models/2012q3/019121.html