MATLAB: Neural Network NAR-based time-series prediction starts failing after several timesteps

closeloopDeep Learning Toolboxnarnetneural networkstime seriestutorial

I am starting to experiment with NAR-based time-series prediction. I've followed several tutorials to write a small simple script to predict a simple sin(t) signal. The resulting prediction is quite good (as expected) at the begining, but as time progresses, the network starts failing catastrophically. Is there anything I am doing wrong?
Here is the code I am using:
DELAY=1:100;
HIDDEN=[10];
t=linspace(1,100,1000);
prueba=cos(t);
datos=prueba;
net = narnet(DELAY,HIDDEN);
[Xs,Xi,Ai,Ts] = preparets(net,{},{},num2cell(datos));
net = train(net,Xs,Ts,Xi,Ai);
net = closeloop(net);
[Xs,Xi,Ai,Ts] = preparets(net,{},{},num2cell(prueba));
y = net(Xs,Xi,Ai);
plot(prueba(DELAY(end)+1:end),'k')
hold on
plot(cell2mat(y),'r')
And the results I am getting are illustrated in the next figure (target-black; prediction-red)

Best Answer

After a not so quick look I have the following comments:
1. cos(t) has a period of 2*pi and satisfies a 2nd order homogeneous difference equation. Therefore
a. Only two delays per period are necessary. No more than eight to sixteen delays per period should be sufficient.
b. No hidden layer is necessary. One hidden layer with one hidden node is rarely better. One hidden layer with H = 2 hidden nodes should be more than sufficient.
2. The autocorrelation function of cos(t) with N = 1000 and dt = 0.1 has 859 positive lag points with significant correlations that have absolute values greater than 0.046.
3. The default divideFcn of narnet is 'dividerand'. However, random sampling of a uniformly sampled time series destroys the beneficial effect of correlations. 'divideblock' and 'divideind' or even 'divideint' with dt = 0.3, should work much better.
4. With I = O =1, N=1000, Ntrn = 700, Nval = 150, Ntst = 150, NFD = 100 and H = 10, there are
Nw = (NFD+1)*H+(H+1)*O = 1021 unknown weights
Ntrneq = Ntrn*O = 700 training equations.
Therefore the net is severely overfit and overtraining mitigation via a large validation set and/or mse regularization should be instituted.
5. Run the original data through the closed loop configuration and separately tabulate trn, val, and tst performance before before testing on new data.
Hope this helps.
Thank you for formally accepting my answer
Greg
Related Question