I have coded up a simple real-value regression RNN in theano.
- What kind of dataset should I test it on?
- How should I go about testing it?
My structure is:
- Univariate (for now) timeseries, $x_{in}(t)$
- $n_{Input Nodes}$ separated by ~equal timesteps, $t_{step}$. Where, $n_{Input Nodes}$ should be sufficiently large to capture a recurrence in the data
- $n_{Hidden Nodes} = n_{Input Nodes}$
- A prediction time lag following the final Input Node of, $lt_{step}$, where $l$ is an integer
- One Output Node taken from the final hidden node, giving a prediction, at $t_{p}=t+lt_{step}$
- $x_{p}(t_p)$ is the prediction of $x_{in}(t_p)$ in training data
- Error by R.M.S.E. $\sqrt{\left(x_{in}(t_p)-x_{p}(t_p)\right)^2}$
- Finally, each node in the hidden layer feeds through to the weight at the next timestep
Tested $y=sin(t) + 0.2*\epsilon$, where $\epsilon \sim N(0,1)$, in a sliding window. I used a historic lag of 5 data points, $y(t-5, t-4, … , t)$, and tried to predict the following point in the curve, $y(t+1)$.
I only used 100 noisey versions of $sin(x)$ over 100 epochs for training. Results weren't too bad…
Thanks for the help. Code seems bug free so I'll optimise for GPU & mini-batches and ramp it up with more up to date algorithms.
Best Answer
A very simple time series to validate the correctness of your code is the one caused by the function sin(x). It's periodic nature makes it a good test function imo. Just print out (or plot) the output activations of your network and compare it with the desired values to see the performance.
Alternatively you can just test XOR like Elman did in his original paper:
101 000 011 110 101 ...