Solved – Forecasting Poisson, accuracy and prediction intervals

count-dataforecastingintermittent time seriesprediction intervaltime series

I'm trying to forecast Poisson data, divided in groups, of 1-26 months of data, depending on the group. Of the pooled data 65% has a value of 0 and 25% a value of 1. I couldn't find any trends or seasonality, so I started to test a couple of different stationairy models. Moving average (3), Moving Average (6), Simple Exponential Smoothing, Naïve and Simple Mean.

I need to forecast 1-6 months ahead and used MAD, MSE and RMSE to test the accuracy of the models. It looks like the most accurate is Simple Mean, with an RMSE of 1 and an MAD of 0,638. I think this is really high but I have no clue how to do anything about this.

Are there forecasting methods I didn't think about that could be way better? Am I over-looking something?

The only thing I was able to find about prediction intervals was F+ts and F-ts with F as forecast, t as t distribution with alfa (n-2) and s as standard deviation. It don't think it was a really trustworthly source but since I wasn't able to find anything else, I'm not sure about how to set up those prediction intervals. Is this method right?

I don't have R to use. I need to do it myself.

Best Answer

You have what is called intermittent demand, that is, a demand time series characterized by "many" zeros. (If your time series is not demand per se, most of what follows will still apply.) So a web search for "forecasting intermittent demand" would already be helpful. Teunter and Duncan (2009, JORS) give an overview of intermittent demand forecasting methods.

The standard method of forecasting intermittent demands is Croston's method. Use exponential smoothing on inter-demand intervals and on nonzero demand sizes separately. The point forecast then is the ratio of the smoothed nonzero demand to the smoothed inter-demand interval. Syntetos and Boylan (2001, IJPE) note that Croston is slightly biased and propose a modification, but this usually doesn't make all that much of a difference in practice.

An alternative is integer autoregressive moving average models (INARMA), which modify the standard ARIMA time series models. Maryam Mohammadipour wrote a thesis on these.

I personally have major doubts about the usefulness of such an expectation point forecast. A time series of 1 demand every other time period has an expectation of 0.5... as does a time series of 2 demands every fourth time period... and so forth - although these are, of course, less and less Poisson-y. I'd argue that it's much more useful to understand the entire future (and predictive) distribution of demands. So I applaud your looking for prediction intervals!

However, the $\alpha(n-2)$ formula you found applies only to single exponential smoothing on continuous data, via the ARIMA model SES is optimal for. So it is inapplicable to count data. I'd much rather propose that you take your point prediction $\hat{y}$ and use quantiles of the Poisson distribution with parameter $\lambda=\hat{y}$. This still disregards parameter estimation uncertainty (along with model selection uncertainty etc.), but it's a simple possibility and likely better than the formula you have.

Shenstone and Hyndman (2005, JoF) note that there is no consistent stochastic model for which Croston's method would be optimal - all candidate models are (1) continuous, not discrete, and (2) can yield negative values. However, for those candidate models, Shenstone and Hyndman provide prediction intervals.

Finally, a word of caution: don't use the MAD for assessing the accuracy of count data forecasts, especially not for intermittent demands. The expected MAD is minimized by the median of your future distribution, not its mean, and if you write that 65% of your data are zeros, then the median is zero... implying that you will probably get the lowest MAD by a flat zero forecast, which is badly biased and likely useless. Here is a presentation I gave at last year's International Symposium on Forecasting on this issue. Or look at Morlidge (2015, Foresight).

Final piece of shameless self-promotion: I have an article in the IJF (Kolassa, 2016) which looks at forecasting low volume count data (mostly intermittent), at different accuracy measures and different forecasting methods, including various flavors of Poisson models. This may be useful to you.

Related Question