MATLAB: Neural Networks extrapolation (using closed network – multistep prediction) can not even predict a line

MATLABmultistep predictionnarneural network

Hi; Is it possible to predict EFFECTIVELY into the future using NAR neural network. I used Neural Networks to predict a simple line using multisteps (closed loop ), but it turns out that it is very good but only till the training part whereas it miserably fails in the multistep (predict ahead ) part. here is my code: Please help, where am I wrong ? %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% MULTISTEP PREDICTION USING NAR AND CLOSED LOOP NEURAL NETWORK
clc; clear all; close all;
% A simple line which I want to predict
DATA= (1:1000)';
%% Create a Nonlinear Autoregressive Network
feedbackDelays = 1:5;
hiddenLayerSize = 5;
net = narnet(feedbackDelays,hiddenLayerSize);
net.divideParam.trainRatio = 85/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 0/100; % NO TEST DATA as I only want to test the multistep part in future
net.divideFcn = 'dividerand'; % Divide data randomly
% TRAINING
N=500; % Number of bars used for training
TDATA = DATA(1:N); % Training Data
T = tonndata(TDATA,false,false);
[x,xi,ai,t] = preparets(net,{},{},T); % prepare data
[net,tr] = train(net,x,t,xi,ai); % Train the Network
y = net(x,xi,ai);
% MULTISPTEP PREDICTION – Closed Loop Network
MSDATA = DATA(N-feedbackDelays(end):end); % multistep predict ahead data
T = tonndata(MSDATA,false,false);
netc = closeloop(net);
[xc,xic,aic,tc] = preparets(netc,{},{},T);
yc = netc(xc,xic,aic);
%% PLot training and multistep predicted part
hold on;
plot(cell2mat(t)); % targets in training part
plot(cell2mat(y),'r'); % nn results in training part
plot([nan(1,length(t)'),cell2mat(tc)],'g'); % targets in multistep predicted part
plot([nan(1,length(y)'),cell2mat(yc)],'r'); % nn in multistep predicted part

Best Answer

Since the target is a straight line, no hidden layer is used. (In general, I would find the minimum number of hidden nodes via a double-for-loop multiple design search).
Testing the CL net on the OL design data for 500 timesteps results in an increase of normalized- mean-square-error from 8e-32 to 5e24 !!!. However, the monotonically increasing CL NMSE only exceeds 0.01 after time = 275.
Similarly, testing the CL net on the nondesign prediction data (501 to 1000 ) yields nmse > 0.01 only after time = 757.
 clc; clear all; close all; plt = 0;
 T  = con2seq(1:1000); d=5
 FD = 1:d; H = []; % H changed to prevent overfitting
 neto = narnet(FD,H);
 neto.divideParam.trainRatio = 85/100;
 neto.divideParam.valRatio   = 15/100;
 neto.divideParam.testRatio  = 0/100; % NOTE 
 % neto.divideFcn = 'dividerand'; Default
 Nd = 500          % Design (Train + Val)
 Td = T(1:Nd);     % Design data
 Tp = T(Nd+1:end); % Prediction data
 Np = length(Tp)   % 500
% OPEN LOOP PERFORMANCE
 [Xo,Xoi,Aoi,To] = preparets(neto,{},{},Td); 
 to= cell2mat(To); varto = var(to,1) % 20419
 rng('default')
 [neto,tro Yo Eo Xof Aof] = train(neto,Xo,To,Xoi,Aoi); 
 yo = cell2mat(Yo);
 NMSEo = mse(Eo)/varto % 8.3362e-32
 plt=plt+1, figure(plt)
 subplot(311), hold on
 plot(to,'LineWidth',2)
 plot(yo,'r--','LineWidth',1)
 title('OPEN LOOP PERFORMANCE')
% CLOSED LOOP PERFORMANCE
 [ netc Xci Aci ]  = closeloop(neto,Xoi,Aoi);
 [ Xc Xci Aci Tc ] = preparets(netc,{},{},Td);
 tc = to;          % isequal(Tc,To) = 1
 [Yc Xcf Acf ]     = netc(Xc,Xci,Aci);
 ec    = cell2mat(gsubtract(Tc,Yc));
 NMSEc = mse(ec)/varto    % 5.3035e+24  WOW!
% Is abs(ec) monotonic?
 u = find(diff(abs(ec) <= 0)) % Empty matrix: 1-by-0
 necsq    = ec.^2/varto;
 cumNMSEc = cumsum(necsq)./(1:Nd-d);
 v        = find(cumNMSEc <= 0.01);
 Nv       = length(v)      % 275 
 NMSEcv   = cumNMSEc(Nv)   % 0.0094645
 yc       = cell2mat(Yc);
 subplot(312), hold on
 plot(tc(v),'LineWidth',2)
 plot(yc(v),'r--','LineWidth',1)
 title('CLOSED LOOP PERFORMANCE')
 subplot(313), 
 plot(ec(v),'LineWidth',2)
 title('CLOSED LOOP ERROR')
% MULTISTEP PREDICTION
 Tm = T(Nd-5:Nd+Nv-1);
 [ Xcm Xcmi Acmi Tcm ] = preparets(netc,{},{},Tm);
 tcm = cell2mat(Tcm); vartcm = var(tcm,1) %  6302
 [Ycm Xcmf Acmf ] = netc(Xcm,Xcmi,Acmi);
 ecm              = cell2mat(gsubtract(Tcm,Ycm));
 NMSEcm           = mse(ecm)/vartcm % 1.3771
% The error amplitude is not monotonic
 u         = find(diff(abs(ecm) <= 0)) %[1,12:19]
 necmsq    = ecm.^2/vartcm;
 cumNMSEcm = cumsum(necmsq)./(1:Nv);
 vm        = find(cumNMSEcm <= 0.01);
 Nvm       = length(vm)       % 257 
 NMSEcmv   = cumNMSEcm(Nvm)   % 0.0090988
 plt = plt+1, figure(plt)
 subplot(211), hold on
 plot(tmc(z),'LineWidth',2)
 plot(ymc(z),'r--','LineWidth',1)
 title('TARGET(Blue) & OUTPUT(Red)')
 subplot(212), 
 plot(emc(z),'LineWidth',2)
Hope this helps.
Thank you for formally accepting my answer
Greg
Related Question