Recently, I have been working on RNNs (LSTM specifically) to do time series prediction and I have used different frameworks such as deeplearning4j and theano (keras). As you may know, one of the hyperparameters of the model is batch_size which has an effect on the model accuracy. I found this paper which suggests that by using online learning instead of batch learning, the convergence can be reached significantly faster using on-line training than batch training, with no apparent difference in accuracy. Now I have the following questions:
- How can we do online learning in any of those models? Does it basically mean to set the batch_size to 1?
- Any ideas and thoughts about this? Is online learning really helpful?
It is worth to mention that I am trying to do time series analysis on streaming data and that is why I thought online learning might be useful in my case.
Best Answer
yes
see Tradeoff batch size vs. number of iterations to train a neural network