I want to predict the next 10 steps for a time series. The single time series (T), has multiple values per time period.
The design criteria, are 1. Forecast accuracy, (next prediction point is more accurate than the 10th prediction point) 2. NN Execution speed. (Training speed is a secondary).
I used the 'Workflow for Neural Network Design' as a template, and the 'Narnet Tutorial On Multi Step Predictions' as a working model.
The code is below. (I specify some default parameters, so that I know what to set them back to when testing different network configurations).
The only relevant (i.e. multi value) dataset I could find for testing is 'oil_dataset'.
The code & results are below:
%%1. Collect & Prepare the data
clear all; clc; plt = 0;PredictSteps = 10;T = oil_dataset;N = length(T);%%2. Create the Neural Network
FD = 2; % Optimize this later with nncorr
H = 10; % Optimize this later with trial/error loops
feedbackDelays = 1:FD; % Default = 1:2
hiddenLayerSize = H; % Default = 10
trainFcn = 'trainbr'; % Default = 'trainlm'
net = narnet(feedbackDelays,hiddenLayerSize,'open',trainFcn);%%3. Configure the network
% Choose Feedback Pre/Post-Processing Functions
net.input.processFcns = {'removeconstantrows','mapminmax'};% Setup dynamic weights & biases
net.adaptFcn = 'adaptwb'; % Default = 'adaptwb'
% Setup Division of Data for Training, Validation, Testing
net.divideFcn = 'divideblock'; % Divide data into blocks
net.divideMode = 'time'; % Divide up every value
net.divideParam.trainRatio = 70/100;net.divideParam.valRatio = 15/100;net.divideParam.testRatio = 15/100;net.trainParam.epochs = 1000; % Default = 1,000
% Choose a Performance Function
net.performFcn = 'mse'; % Mean squared error
% For MSE, Normalize errors of multiple outputs in different ranges
net.performParam.normalization = 'standard';% Choose Plot Functions
net.plotFcns = {'plotperform','plottrainstate','plotresponse', ... 'ploterrcorr','plotinerrcorr','plotwb','ploterrhist','plotfit', ... 'plotregression',};%%4. Initialize weights & biases
rng('default');% In this case set 'weighted Error Weights'
ind = 1:N; ew = 0.99.^(N-ind); % Plot the Error Weights
plt = plt + 1;figure(plt), hold onplot(ew)title('Error Weights');ew = con2seq(ew);
%%5.a) Train the network
[Xo,Xio,Aio,To,EWo] = preparets(net,{},{},T, ew);% Plot original data
to = cell2mat(To);plt = plt + 1;figure(plt), hold onplot(FD+1:N, to)legend('Upper', 'Lower')title('INITIAL TRAINING DATA');
% Train the NN
[net,tr,Yo,Eo,Afo,Xfo] = train(net,Xo,To,Xio,Aio,EWo);%%5.b) Plot the OPENLOOP traing results
yo = cell2mat(Yo);plt = plt + 1;figure(plt), hold onplot(FD+1:N, to, 'b') plot(FD+1:N, yo, 'r--')legend('TARGET', 'OUTPUT') % Fix the legend colors, need to make legend for each line
title('OPENLOOP NARNET RESULTS');
%%5.c) Validate the trained 'Open' Network
[Yo, Xfo, Afo] = net(Xo,Xio,Aio);Eo = gsubtract(To,Yo);OpenLoopMSEperformance = mse(net, To, Yo); % 728.0917
OpenLoopPERFORMperformance = perform(net,To,Yo, EWo); % 0.0063
% NMSEopen = mse((Eo)/var(To,1)); % Error: Undefined function 'sum' for input arguments of type 'cell'
view(net);%%5.d) Close the Network
[netc, Xic,Aic] = closeloop(net,Xio,Aio);netc.name = [net.name ' - Closed Loop'];view(netc);[Xc,Xic,Aic,Tc,EWc] = preparets(netc,{},{},T,ew); % should we use EWo instead of ew?
[Yc,Xfc,Afc] = netc(Xc,Xic,Aic);%%5.e) Plot the CLOSELOOP results
tc = cell2mat(Tc); % tc isequal to
yc = cell2mat(Yc);plt = plt + 1;figure(plt), hold onplot(FD+1:N, tc, 'b') plot(FD+1:N, yc, 'r--')legend('TARGET', 'OUTPUT') % Fix the legend colors
title('CLOSELOOP NARNET RESULTS')
%%5.f) Validate the closed network
Ec = gsubtract(Tc,Yc);CloseLoopMSEperformance = mse(netc, Tc, Yc); % 9.8945e+04
CloseLoopPERFORMperformance = perform(netc,Tc,Yc,EWc); % 0.5823
% NMSEclosed = mse((Ec)/var(Tc,1)); % Error: Undefined function 'sum' for input arguments of type 'cell'
view(netc);%%5.g) ReTrain the 'Closed' network for better accuracy
% 'preparets' performed in section 5.d)
[netc2, trc, Yc2, Ec2, Xfc2, Afc2] = train(netc, Xc, Tc, Xic, Aic, EWc);[Yc3,Xfc3, Afc3] = netc2(Xc,Xfc2,Afc2);%%5.h) Plot the retrained CLOSELOOP results
yc3 = cell2mat(Yc3);% tc = to % tc isequal to
plt = plt + 1;figure(plt), hold onplot(FD+1:N, tc, 'b') plot(FD+1:N, yc3, 'g--')legend('TARGET', 'OUTPUT') % Fix the legend colorstitle('TRAINED CLOSELOOP NARNET RESULTS');
%%6. Validate the trained 'Closed' Network
Ec2 = gsubtract(Tc,Yc3);TrainedClosedLoopMSEperformance = mse(netc2, Tc, Yc3); % 6.3859e+04
TrainedClosedLoopPERFORMperformance = perform(netc2,Tc,Yc3,EWc); % 0.2147
% NMSEs = mse((Ec2)/var(Tc,1)); % Error: Undefined function 'sum' for input arguments of type 'cell'
view(netc2);%%7.a) Use the network (for multi-step prediction)
Xc3 = cell(1, PredictSteps); % Empty cells for prediction
[Yc4,Xfc4,Afc4] = netc2(Xc3,Xfc3,Afc3);%%7.b) Plot the predicted results
yc4 = cell2mat(Yc4);plt = plt+1; figure(plt), hold onplot(FD+1:N, tc)plot(FD+1:N, yc3)plot(N+1:N+PredictSteps, yc4, '+')legend('TARGET', 'OUTPUT' , 'TARGETLESS PREDICTION')title('NARNET PREDICTION RESULTS');
%%Save the NN and the final input and layer delay states, Xfc and Afc
Xis = Xfc4;Ais = Afc4;save('NARNETfcn.mat', 'Xis', 'Ais'); % Ready for use next time
As you can see, once the narnet is closed, the performance is bizarre.
I ran the 'Narnet Tutorial On Multistep Ahead Predictions' on oil_dataset and produced similar results.
1. What am I doing wrong?
2. How do I correct the error calculating NMSEs?
Thank you in anticipation.
Best Answer