Solved – How to improve the accuracy of an ARIMA model

arimaforecastingmachine learningtime series

Im currently developing a simple ARIMA model to forecast a time-series data. Unfortunately my model is not providing good results.

Ive checked if the data is stationary through Augmented Dickey-Fuller Test. Came up as stationary (P< 0,05)

I used auto.arima to verify the p,d,q values, and it provided (2,1,1).

The dataset has 201 data points and the time-series is measured monthly. I would like to forecast for the next 12 months, although Im still getting very bad accuracy results:

             ME     RMSE       MAE       MPE     MAPE

Test set 0.06804923 0.348731 0.2659965 -73.86601 140.3297

Why is my MAPE over 100?
How can I improve such accuracy, am I missing any step to perform a successful ARIMA model?

DATASET:

structure(c(0.52, 0.36, 0.6, 0.8, 0.21, 0.42, 1.19, 0.65, 0.72,
1.31, 3.02, 2.1, 2.25, 1.57, 1.23, 0.97, 0.61, -0.15, 0.2, 0.34,
0.78, 0.29, 0.34, 0.52, 0.76, 0.61, 0.47, 0.37, 0.51, 0.71, 0.91,
0.69, 0.33, 0.44, 0.69, 0.86, 0.58, 0.59, 0.61, 0.87, 0.49, -0.02,
0.25, 0.17, 0.35, 0.75, 0.55, 0.36, 0.59, 0.41, 0.43, 0.21, 0.1,
-0.21, 0.19, 0.05, 0.21, 0.33, 0.31, 0.48, 0.44, 0.44, 0.37,
0.25, 0.28, 0.28, 0.24, 0.47, 0.18, 0.3, 0.38, 0.74, 0.54, 0.49,
0.48, 0.55, 0.79, 0.74, 0.53, 0.28, 0.26, 0.45, 0.36, 0.28, 0.48,
0.55, 0.2, 0.48, 0.47, 0.36, 0.24, 0.15, 0.24, 0.28, 0.41, 0.37,
0.75, 0.78, 0.52, 0.57, 0.43, 0, 0.01, 0.04, 0.45, 0.75, 0.83,
0.63, 0.83, 0.8, 0.79, 0.77, 0.47, 0.15, 0.16, 0.37, 0.53, 0.43,
0.52, 0.5, 0.56, 0.45, 0.21, 0.64, 0.36, 0.08, 0.43, 0.41, 0.57,
0.59, 0.6, 0.79, 0.86, 0.6, 0.47, 0.55, 0.37, 0.26, 0.03, 0.24,
0.35, 0.57, 0.54, 0.92, 0.55, 0.69, 0.92, 0.67, 0.46, 0.4, 0.01,
0.25, 0.57, 0.42, 0.51, 0.78, 1.24, 1.22, 1.32, 0.71, 0.74, 0.79,
0.62, 0.22, 0.54, 0.82, 1.01, 0.96, 1.27, 0.9, 0.43, 0.61, 0.78,
0.35, 0.52, 0.44, 0.08, 0.26, 0.18, 0.3, 0.38, 0.33, 0.25, 0.14,
0.31, -0.23, 0.24, 0.19, 0.16, 0.42, 0.28, 0.44, 0.29, 0.32,
0.09, 0.22, 0.4, 1.26, 0.33, -0.09, 0.48), .Tsp = c(1, 17.6666666666667,
12), class = "ts")

Thanks

Best Answer

I took your 201 monthly values and examined them in an automatic mode with AUTOBOX , a time series analysis package that I helped to develop. The model developed is here . The Actual,Fit and Forecast graph is here and the ACF of the residuals is here . The forecast plot for the next 12 periods is here. .

There are a number of anomalous data points that will thwart any simple brute force attempt to examine candidate arima structure . auto.arima as it is not robust to latent anomalies (pulses,step shifts, seasonal pulses, local time trends ) and thusly over differences and over paramaterizes arima structure induced by the over differencing.

In terms of your "complaint" about the size of out-of-sample mapes , it all has to do with forecasting "small numbers" as was pointed out by @jbowman .

Finally the plot of the model residuals markedly more visually acceptable than yours.

In closing .... to improve the accuracy of any model "Know the Assumptions" as A. Wald wisely reflected.

Related Solutions

Solved – How to dampen forecast to improve accuracy

Dampening can be thought of as a special case of shrinkage methods; these methods as a whole tend to reduce uncertainty in estimates (yet another circumstance of trading bias for variance, an ever-recurring theme in statistics, though in some cases, such as many involving variable-selection, shrinkage can reduce both bias and variance).

There are many methods by which forecasts are produced, but they have a wide variety of characteristics.

Consider, for example, a case where we just observe independent random values with a constant mean. Then dampening would involve moving our forecasts a little toward zero from the sample mean.

In methods that estimate a local-mean effect element (some form of adaptive level, say), dampening could move the forecast of that recent level toward 0 (or, for slightly different models, toward the overall mean), so that forecast trends tended to 'fall back' from the most recent excursion over longer forecast horizons.

In methods that have an element of recent local linear trend (some form of adaptive trend, say), dampening could move the forecast of that recent trend toward 0, so that forecast trends tended to 'flatten out' over longer forecast horizons.

Now consider a case where, for example, there's seasonal effects around some other trend - dampening of the seasonal component would tend to shrink the forecast strength of the seasonality, "smoothing out" the wiggle over longer forecast horizons.

The text by Hyndman and Athanasopoulos, Forecasting principles and practice, (freely available on-line as well as in dead-tree form) has a section on dampening, but you need some of the preceding sections for context - the models in sec 7.4 are damped versions of the models in sec 7.2, Holt linear trend models). I highly recommend investigating this book. [You may find the older book on forecasting by Makridakis, Wheelwright and Hyndman in libraries or on forecaster's bookshelves. It's also very handy.]

That section of text on dampening includes something like the kind of thing you're asking for - a tunable smoothing parameter, $\phi$, between $0$ and $1$, to produce a dampening effect ... but as you'll see if you look at that section, it's of a different form for different models - the additive trend and multiplicative trend models use different formulas!

So how might we "dampen" a linear least squares fit*?

*(Note that I don't particularly regard this as a suitable approach for very many forecasting problems, but the Hyndman and Athanasopoulos text is a better place to investigate the merits of various approaches. Nevertheless, let us proceed, since a discussion of the issues covers many of the issues one must consider when trying to dampen models more generally)

A linear regression doesn't have any 'local' component to it at all, it's a global model. In any case, as I mentioned at the start, dampening can be thought of as shrinkage, and we can still shrink that estimate of the linear component toward 0.

But we then have the question of about which center do we 'tilt' the line? The obvious approach for an actually global model would be to do it about the mean:

The line in center-slope form would be $y-\bar{y} = \beta (x-\bar{x})+\varepsilon_t$.

The point forecasts are $\hat{y_t} = \bar{y} + b (x_t-\bar{x})$, where $b=\hat{\beta}$, the slope estimate.

If $T$ is the last observed time, then the point forecasts are

$\hat{y_{T+k}} = \bar{y} + b (x_{T+k}-\bar{x})$.

One simple and commonly used dampening is to multiply it at each step by a constant $\phi$, where $0<\phi<1$ which would let the slope shrink with time:

$\hat{y_{T+k}} = \bar{y} + b_{T+k} (x_{T+k}-\bar{x})$,

where $b_{T+k} = \phi b_{T+k-1} = \phi^k b_{T} = \phi^k b$.

enter image description here

However, as I said, a global model doesn't necessarily make as much sense as some other options if it's a situation in which we'd want to apply such dampening; we might consider instead pivoting the slope around a point somewhere near the end (nearer to $t=T$), to make it more local... but if we think the model should be local in that sense, we should probably be looking at locally-linear models to start with, and then dampen those trends.

There's some distinct similarities in general approach I took here to the one used for the additive trend model, though that's a local model, not a global one - and thereby more suited to forecasting trends that tend to be linear for a while, but where the linear trend isn't constant in the long term.

That approach of progressive multiplication of a model-component by $\phi$ could be used for any number of models if applied to the parts it makes sense to shrink to be smaller - it could be applied to seasonal components, or AR parameters, or any number of other things. Different components of forecasting models may even be shrunk at different rates.

[That progressive multiplication by a constant (geometric shrinkage) isn't the only way to shrink components of a model, but it's the most common.]

Solved – Improve ARIMAX model, compared to arima model

Look at your xreg matrix and you should spot the problem. The following code does what you are trying to do more simply.

library(forecast)
xreg <- seasonaldummies(ts0enb)
fit <- auto.arima(ts0enb, xreg=xreg)
accuracy(fit)

It does not work as well as the standard ARIMA model because the seasonality is not strong (or perhaps it changes so much over the time period that it is difficult to model).

Best Answer

Related Solutions

Solved – How to dampen forecast to improve accuracy

Solved – Improve ARIMAX model, compared to arima model

Related Question