Solved – How to interpret BIC

bicfittingmodel

I am fitting two different models to the same data. In one model, there is one free parameter for three different experimental conditions. In another model, I fit three free parameters, one for each condition. I do this for 10 subjects in a dataset.

For each subject, the model with fewer free parameters has a higher BIC. But for every single subject, the difference in BIC is roughly the same (about 10). I find this very suspicious, since the BIC values themselves range from ~30 to ~1000.

I have never used BIC before, and would like to say that the model with one free parameter is better.

Best Answer

Given your further comment, I am not surprised at this result. BIC is a penalized log likelihood. It is useful for comparing models on one data set (here, each participant), but not for comparing across data sets.

What this result is telling you, in essence, is that the model fits very differently for different people, but that the amount of improvement in the fit by adding two parameters is about the same for each person.

Related Question