Neural Networks – How to Improve Sine Wave Predictions

kerasneural networkspythonregression

Here, have a look:
Generated Sine Wave
You can see exactly where the training data ends. Training data goes from $-1$ to $1$.

I used Keras and a 1-100-100-2 dense network with tanh activation.
I calculate the result from two values, p and q as p / q. This way I can achive any size of number using only smaller than 1 values.

Please note I am still a beginner in this field, so go easy on me.

Best Answer

You're using a feed-forward network; the other answers are correct that FFNNs are not great at extrapolation beyond the range of the training data.

However, since the data has a periodic quality, the problem may be amenable to modeling with an LSTM. LSTMs are a variety of neural network cell that operate on sequences, and have a "memory" about what they have "seen" before. The abstract of this book chapter suggests an LSTM approach is a qualified success on periodic problems.

In this case, the training data would be a sequence of tuples $(x_i, \sin(x_i))$, and the task to make accurate predictions for new inputs $x_{i+1} \dots x_{i+n}$ for some $n$ and $i$ indexes some increasing sequence. The length of each input sequence, the width of the interval which they cover, and their spacing, are up to you. Intuitively, I'd expect a regular grid covering 1 period to be a good place to start, with training sequences covering a wide range of values, rather than restricted to some interval.

(Jimenez-Guarneros, Magdiel and Gomez-Gil, Pilar and Fonseca-Delgado, Rigoberto and Ramirez-Cortes, Manuel and Alarcon-Aquino, Vicente, "Long-Term Prediction of a Sine Function Using a LSTM Neural Network", in Nature-Inspired Design of Hybrid Intelligent Systems)