I understand the main idea of the expression of the AIC and BIC however, I do not understand how they work. I have a small data mtcars from R program. When, I calculate the AIC and BIC, I found that the values of them is high (636 and 660), respectively). I wonder how I got this values while my data is very small. So, does the values of AIC and BIC increase as the data increases? Do they depends on the number of observation? Do I did something wrong?
I did this:
1- Estimate the mixture weights ($\pi$).
2- Estimate the values of the model parameters.
3- Then, calculate the expectation logliklihood function (I am using EM-method).
4- Then, calculate the AIC and BIC.
The logliklihood:
$ll = sum(\tau_1*(log (\pi_1) +log(density_1)) + \tau_2*(log (\pi_2) + log(density_2)))$
$AIC = -2*ll +2k$
$BIC = -2*ll + log(N)*k$
Best Answer
The value of AIC and BIC depend on the sample size because the likelihood does (it is a product of a number of terms which depends on the sample size: remember the density you plug in your $ll$ formula is the joint density of the sample). Further, in BIC you have the term log(N) which implies a heavier penalty for parameters in BIC than in AIC if the sample size is large (as log(N) grows larger than 2).