Solved – Different AIC values in trace and final output in `auto.arima`

aicarimar

I am trying to fit a time series with function auto.arima in the "forecast" package in R that is choosing the best model automatically. Since I am using it for my thesis, I have to show the trace of stepwise selection algorithm manually (without adding line "trace" in auto.arima code) with my web apps here https://nugraha92.shinyapps.io/Analisis/

But, when I manually checked it, I found that auto.arima did not choose minimum AIC values.

Then, I input the trace=TRUE line and the result shows that the AIC/BIC value is not the same between the "trace ARIMA list" and "choosen ARIMA model". For example, in the image it is shown that the best model is ARIMA(3,1,3), BIC score at the list is -252.4451 while at the equation below is -250.89.

Why and which one is the correct one? Did I make some mistakes?

Best Answer

auto.arima uses an approximation to speed up calculations. The final reported AIC/AICc/BIC values are computed without the approximation. Set approximation=FALSE if you want to avoid using the approximation. Then the values produced by trace=TRUE will match the final values reported.

Related Solutions

Solved – auto.arima from Forecast package

arima() will return different results from auto.arima() when differencing is used. Estimation with arima() uses a diffuse prior when the model is non-stationary, whereas auto.arima() fits the model to the differenced series.

I'm not sure why the two auto.arima() calls give different AIC values for the same model. Can you please post the data so the result can be replicated.

Solved – R time-series forecasting with neural network, auto.arima and ets

In-sample fits are not a reliable guide to out-of-sample forecasting accuracy. The gold standard in forecasting accuracy measurement is to use a holdout sample. Remove the last 30 days from the training sample, fit your models to the rest of the data, use the fitted models to forecast the holdout sample and simply compare accuracies on the holdout, using Mean Absolute Deviations (MAD) or weighted Mean Absolute Percentage Errors (wMAPEs).

Here is an example using R. I am using the 2000th series of the M3 competition, which already is divided into the training series M3[[2000]]$x and the test data M3[[2000]]$xx. This is monthly data. The last two lines output the wMAPE of the forecasts from the two models, and we see here that the ARIMA model (wMAPE 18.6%) outperforms the automatically fitted ETS model (32.4%):

library(forecast)
library(Mcomp)

M3[[2000]]

ets.model <- ets(M3[[2000]]$x)
    arima.model <- auto.arima(M3[[2000]]$x)

ets.forecast <- forecast(ets.model,M3[[2000]]$h)$mean
arima.forecast <- forecast(arima.model,M3[[2000]]$h)$mean

sum(abs(ets.forecast-M3[[2000]]$xx))/sum(M3[[2000]]$xx)
sum(abs(arima.forecast-M3[[2000]]$xx))/sum(M3[[2000]]$xx)

In addition, it looks like there are abnormally high sales near indices 280-300. Could this be Christmas sales? If you know about calendar events like these, it would be best to feed those to your forecasting model as explanatory variables, which will give you a better forecast next time that Christmas rolls around. You can do that easily in ARIMA(X) and NNs, not so easily in ETS.

Finally, I recommend this textbook on forecasting: http://otexts.com/fpp/

Best Answer

Related Solutions

Solved – auto.arima from Forecast package

Solved – R time-series forecasting with neural network, auto.arima and ets

Related Question