Solved – Significant difference between AIC in generalized mixed models

aicglmmrspss

I have fit several generalized mixed models for a multinomial distribution data in SPSS.

I want to select the best one for explaining my data. But comparing the AIC and selecting the lower AIC model gives me a model without sense and its accuracy is lower than others with higher AIC.

I want to test if there is any statistical difference between the AIC of the different models, because maybe the values are not statistically different and I could select another model with higher accuracy. Do you know how to do that? E.g.: AICa= 1193 (accuracy 81,3%), AICb= 1273 (88,5%)

I know that with R you can test the differences between models, but I can't develop a generalized mixed model with multinomial function in R. Is that possible in SPSS?

Best Answer

Don't choose a model just because it has a better AIC or a better AICc or a better $R^2$ or any other better property if that model doesn't make any sense.

Model selection is an art. It requires a balance of statistical knowledge and substantive knowledge.

Statistics, more generally, is part of a reasoned argument for or against certain propositions. It ought to be designed to improve knowledge, in whatever field you are in and whether this involves exploration, modeling, or whatever.

Part of the point of learning a lot of statistics is to be able to answer interesting questions and make stronger arguments. It is not to let the computer do your thinking for you.

Related Solutions

Solved – Using AIC to select between models that use nested and non-nested variables

According to an informal document by Burnham, who I regard as one of the leading experts on the AIC, the notion that models need to be nested to use the AIC for model comparison is a myth. Here is the pdf. See item number 2.

While we're on the topic, I might suggest using the AICc instead of the AIC, as Burnham and Anderson (2004) recommend it as a better default model selection strategy due to its bias correction for finite samples.

AIC – Comparing AIC Between GLM and GAM Models

Yes, they are comparable, but you shouldn't be using REML to compare models with different fixed effects. Use method = "ML" in the gam() call if you are comparing your polynomial fits with the smooth version.

You could fit your GLM via gam() instead and as there are no penalised terms it would be fitted using the same algorithm as used by glm().

r$> m_glm <- glm(y ~ poly(x1, 2, raw = TRUE) + x2, family = binomial, data = df)
r$> m_glm_2 <- gam(y ~ poly(x1, 2, raw = TRUE) + x2,
                     family = binomial, data = df, method = "ML")
r$> m_gam <- gam(y ~ s(x1) + s(x2),
                    family = binomial, data = df, method = "ML")           
r$> AIC(m_glm, m_glm_2, m_gam)            
              df      AIC
m_glm   4.000000 75.72616
m_glm_2 4.000000 75.72616
m_gam   5.072922 78.50021

As there may be slight implementation differences, I would fit the GLM via gam(), and note that AIC() for a GAM in mgcv is going to be correcting for the smoothing parameters having been estimated, which involves a correction to the EDF of the model (though this is taken care of for you via the logLik() method for "gam" classed objects.

Best Answer

Related Solutions

Solved – Using AIC to select between models that use nested and non-nested variables

AIC – Comparing AIC Between GLM and GAM Models

Related Question