Chris Chatfield, whose many quality books and papers I enjoyed reading, in (1) gives the following advice:
For example, the choice between ARIMA time-series models with low and approximately equal values of the AIC should probably be made, not on which happens to give the minimum AIC, but on which gives the best forecasts of the most recent year's data.
What is the rationale for such advice? If it is sound, why does forecast::auto.arima
and other forecasting routines do not follow it? Yet to be implemented? It has already been discussed here that to look for models that just happened to give the minimum AIC is probably not a good idea. Why is the option to have $n\ge1$ ARIMA models with low but approximately equal (e.g. within 1 or 2 values of the minimal AIC) is not a default in much of the time series forecasting software?
(1) Chatfield, C. (1991). Avoiding statistical pitfalls. Statistical Science, 6(3), 240–252. Available online, URL: https://projecteuclid.org/euclid.ss/1177011686.
Best Answer
It's true that if you have multiple AIC values approximately equal selecting the lowest value may be not the best option. A sensible alternative would be performing model averaging. This way you are able to use not just the best model for inference, but a set of "most supported" models each one weighted according to their AIC value.
You have a short introduction by Vincent Calcagno here