Solved – Time Series Hold Out Data not used to build model

forecastingrtime series

It is my understanding that if one wants to build multiple time series models on a time series that goes from 2000 to today (2015) monthly; and one wanted to use that information to forecast 3 months in the future, it is common approach to split your data into "train" and "test" datasets.

Your test dataset would be the last 3 months of your time series (jan 2015, feb 2015, march 2015) (pretend we're already in april for simplicity sake). You would then 'define' your model on your training dataset, and then compute it's errors against your test dataset (defined as predicted vs actual).

This way you could try out many multiple models and pick the one with the lowest "forecast prediction error".

However my question is: By ignorning those last 3 months of data, how do you then use that model to forecast values later in time? Example: say you wanted to forecast April-June. Is it standard procedure to apply the same model (that wasn't built on the last 3 months) to the April – June forecast period?

If so is this something you can do in R with a package? It seems like the forecast function only works to forecast forward from the training dataset, and you can't apply it to other time series objects.

Or does one 're-build' the model on the entire time series (2000 to 2015 March) and then use that model to forecast into April-June?

I am pretty confused by this and any help would be appreciated.

Best Answer

Well, technically you should have your model ex ante determined, and the so-called train data set is really just to demonstrate the universal validity of your model on "the present day" (test) data set. It is a subsample test.

Once the model you're using is shown to work robustly as a predictor in all important subsamples, and you've argued convincingly that it is universally valid, it's typical to run the whole thing again using all the data and try to predict the future. This part usually tends to be the problematic part.

Related Question