Solved – ARIMA: How to interpret MAPE

accuracyarimaforecastingmase

I am using the forecast package in R to generate an ARIMA model for my data. I started with the auto.arima function for a try and got a ARIMA(1,1,2) model.

         ar1      ma1     ma2
      0.7734  -1.0773  0.1191
s.e.  0.0709   0.0962  0.0824

But my question is not about the model itself but more about the validation of the model accuracy in general. Therefore I used "accuracy(fit)" and obtained the following output:

                    ME     RMSE      MAE      MPE     MAPE     MASE        ACF1
Training set -1.580214 163.8034 94.91732 -4.18724 13.61585 1.029359 0.002118006

I interpreted the MAPE like, "on average, the forecast if off by 14%", which sounds fine for me. But on the other side the MASE is greater than 1, which means the model is worse than a naive model. (?)

When I plot a forecast obviously the model is really bad:

fc<-forecast(fit, level=80, h=100)
plot(fc)

arima

I am confused about the interpretation of the accuracy parameters I got. Did I miss some R command or how can I interpret the accuracy? Thanks!

Best Answer

No, MASE>1 does not mean the model's forecast is worse than a naive forecast. The reference forecast in the denominator of MASE is the in-sample naive model's error, while in the numerator you have the actual forecast's (out-of-sample) error. Check out the formula here. See also this question and answer for more details.

Whether the model is really bad is a relative question; are you sure any other model can do much better? Some things are just inherently difficult to forecast.

Related Question