I am doing some forecasting in R using Rob Hyndman's forecast package. The paper belonging to the package can be found here.
In the paper, after explaining the automatic forecasting algorithms, the authors implement the algorithms on the same data set. However, after estimating both an exponential smoothing and ARIMA model they make a statement I do no understand (on page 17):
Note that the information criteria are not comparable.
I thought that an advantage of using AIC for model selection is that we can compare AIC values from different models, as long as they are estimated using the same data set. Is this incorrect?
This matter is of particular interest to me, as I was planning on combining forecasts from different model classes (e.g. exponential smoothing and ARIMA) using so called Akaike weights (see Burnham and Anderson, 2002, for discussion on Akaike weights)
References
- Burnham, K. P., & Anderson, D. R. (2002). Model selection and multi-model inference: a practical information-theoretic approach. Springer Verlag.
Best Answer
The two models treat initial values differently. For example, after differencing, an ARIMA model is computed on fewer observations, whereas an ETS model is always computed on the full set of data. Even when the models are equivalent (e.g., an ARIMA(0,1,1) and an ETS(A,N,N)), the AIC values will be different.
Effectively, the likelihood of an ETS model is conditional on the initial state vector, whereas the likelihood of a non-stationary ARIMA model is conditional on the first few observations, even when a diffuse prior is used for the nonstationary components.