I'm looking into using a LSTM (long short-term memory) version of a recurrent neural network (RNN) for modeling timeseries data. As the sequence length of the data increases, the complexity of the network increases. I am therefore curious what length of sequences would be feasible to model with a good accuracy?
I would like to use a relatively simple version of the LSTM without any difficult to implement state-of-the-art approaches. Each observation in my timeseries would likely have 4 numeric variables and the number of observations would be around 100.000 to 1.000.000.
Best Answer
It totally depends on the nature of your data and the inner correlations, there is no rule of thumb. However, given that you have a large amount of data a 2-layer LSTM can model a large body of time series problems / benchmarks.
Furthermore, you don't backpropagate-through-time to the whole series but usually to (200-300) last steps. To find the optimal value you can cross-validate using grid search or bayesian optimisation. Furthermore, you can have a look at the parameters here: https://github.com/wojzaremba/lstm/blob/master/main.lua.
So, the sequence length doesn't really affect your model training but it's like having more training examples, that you just keep the previous state instead of resetting it.