Time Series Forecasting – Using Neural Networks for Time Series Forecasting

forecastingneural networkstime series

I'm new to machine learning, and I have been trying to figure out how to apply neural network to time series forecasting. I have found resource related to my query, but I seem to still be a bit lost. I think a basic explanation without too much detail would help.

Let's say I have some price values for each month over a few years, and I want to predict new price values. I could get a list of prices for the last few months, and then try to find similar trends in the past using K-Nearest-Neighbor. I could them use the rate of change or some other property of the past trends to try and predict new prices. How I can apply neural network to this same problem is what I am trying to find out.

Best Answer

Here is a simple recipe that may help you get started writing code and testing ideas...

Let's assume you have monthly data recorded over several years, so you have 36 values. Let's also assume that you only care about predicting one month (value) in advance.

  1. Exploratory data analysis: Apply some of the traditional time series analysis methods to estimate the lag dependence in the data (e.g. auto-correlation and partial auto-correlation plots, transformations, differencing). Let's say that you find a given month's value is correlated with the past three month's data but not much so beyond that.
  2. Partition your data into training and validation sets: Take the first 24 points as your training values and the remaining points as the validation set.
  3. Create the neural network layout: You'll take the past three month's values as inputs and you want to predict the next month's value. So, you need a neural network with an input layer containing three nodes and an output layer containing one node. You should probably have a hidden layer with at least a couple of nodes. Unfortunately, picking the number of hidden layers, and their respective number of nodes, is not something for which there are clear guidelines. I'd start small, like 3:2:1.
  4. Create the training patterns: Each training pattern will be four values, with the first three corresponding to the input nodes and the last one defining what the correct value is for the output node. For example, if your training data are values $$x_1,x_2\dots,x_{24}$$ then $$pattern 1: x_1,x_2,x_3,x_4$$ $$pattern 2: x_2,x_3,x_4,x_5$$ $$\dots$$ $$pattern 21: x_{21},x_{22},x_{23},x_{24}$$
  5. Train the neural network on these patterns
  6. Test the network on the validation set (months 25-36): Here you will pass in the three values the neural network needs for the input layer and see what the output node gets set to. So, to see how well the trained neural network can predict month 32's value you'll pass in values for months 29, 30, and 31

This recipe is obviously high level and you may scratch your head at first when trying to map your context into different software libraries/programs. But, hopefully this sketches out the main point: you need to create training patterns that reasonably contain the correlation structure of the series you are trying to forecast. And whether you do the forecasting with a neural network or an ARIMA model, the exploratory work to determine what that structure is is often the most time consuming and difficult part.

In my experience, neural networks can provide great classification and forecasting functionality but setting them up can be time consuming. In the example above, you may find that 21 training patterns is not enough; different input data transformations lead to a better/worse forecasts; varying the number of hidden layers and hidden layer nodes greatly affects forecasts; etc.

I highly recommend looking at the neural_forecasting website, which contains tons of information on neural network forecasting competitions. The Motivations page is especially useful.

Related Question