MATLAB: Neural network test with a new data set

data structureneural networktest with a new data

Dear Matlab experts. Actaully, I'm not familiar with neural network analysis. I want to forecast outdoor air temperature with input set(ground temp, cloud, relative humidity). The training/validation/testing is okay with the input data set(1X2877) and target data(1X2877). However, I have trapped to evaluate the network with the new data set (1X960) (same input style). Would you light on for me? I'm lost my way to resolve the issue. I apprecaite your valuable time to concern on this issue.
===here is my code===
% Solve an Autoregression Problem with External Input with a NARX Neural Network % Script generated by NTSTOOL % Created Fri Feb 22 15:22:18 EST 2013 % % This script assumes these variables are defined: % % JULYTH – input time series. % JULYE – feedback time series.
%This is 1X2877 matrix data has [a;b;c] for each
inputSeries = tonndata(JYTH,false,false);
% This is 1X2877 matrix data has target output targetSeries = tonndata(JYE,false,false);
% Create a Nonlinear Autoregressive Network with External Input
inputDelays = 1:2;
feedbackDelays = 1:2;
hiddenLayerSize = 10;
net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
% Choose Input and Feedback Pre/Post-Processing Functions % Settings for feedback input are automatically applied to feedback output % For a list of all processing functions type: help nnprocess % Customize input parameters at: net.inputs{i}.processParam % Customize output parameters at: net.outputs{i}.processParam
net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'};
net.inputs{2}.processFcns = {'removeconstantrows','mapminmax'};
%net.inputs{3}.processFcns = {'removeconstantrows','mapminmax'};
% Prepare the Data for Training and Simulation % The function PREPARETS prepares timeseries data for a particular network, % shifting time by the minimum amount to fill input states and layer states. % Using PREPARETS allows you to keep your original time series data unchanged, while % easily customizing it for networks with differing numbers of delays, with % open loop or closed loop feedback modes.
[inputs,inputStates,layerStates,targets] = preparets(net,inputSeries,{},targetSeries);
% Setup Division of Data for Training, Validation, Testing % The function DIVIDERAND randomly assigns target values to training, % validation and test sets during training. % For a list of all data division functions type: help nndivide
net.divideFcn = 'dividerand'; % Divide data randomly
% The property DIVIDEMODE set to TIMESTEP means that targets are divided % into training, validation and test sets according to timesteps. % For a list of data division modes type: help nntype_data_division_mode
net.divideMode = 'value'; % Divide up every value
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
% Choose a Training Function % For a list of all training functions type: help nntrain % Customize training parameters at: net.trainParam
net.trainFcn = 'trainlm'; % Levenberg-Marquardt
% Choose a Performance Function % For a list of all performance functions type: help nnperformance % Customize performance parameters at: net.performParam
net.performFcn = 'mse'; % Mean squared error
% Choose Plot Functions
% For a list of all plot functions type: help nnplot % Customize plot parameters at: net.plotParam
net.plotFcns = {'plotperform','plottrainstate','plotresponse', … 'ploterrcorr', 'plotinerrcorr'};
% Train the Network
[net,tr] = train(net,inputs,targets,inputStates,layerStates);
% Test the Network
outputs = net(inputs,inputStates,layerStates);
errors = gsubtract(targets,outputs);
performance = perform(net,targets,outputs)
% Recalculate Training, Validation and Test Performance
trainTargets = gmultiply(targets,tr.trainMask);
valTargets = gmultiply(targets,tr.valMask);
testTargets = gmultiply(targets,tr.testMask);
trainPerformance = perform(net,trainTargets,outputs)
valPerformance = perform(net,valTargets,outputs)
testPerformance = perform(net,testTargets,outputs)
% View the Network
view(net)
_% From this part I want to run a new test or forecast with new inputs % This is a new inputs 1X960. The maxrix has the same structure for the % testing [a;b;c]
inputSeries2 = tonndata(AUGTH,false,false);
[inputs2,inputStates2,layerStates2,targets2] = preparets(net,inputSeries2);
% When I want to generate a new output from the network all "output2"(1X960) has % NaN. I suspect that "inputStates2" has NaN value its second row. Would % you let me know how I resolve the issue and get the new output2?
outputs2 = net(inputs2,inputStates2,layerStates2);_

Best Answer

I want to forecast outdoor air temperature with input set(ground temp, cloud, relative humidity). The training/validation/testing is okay with the input data set(1X2877) and target data(1X2877).
What, exactly, does "okay" mean?... What are the val and test R^2 values?
How can you have a one dimensional input data set when you have 3 input variables?
This script assumes these variables are defined: JULYTH - input time series. JULYE - feedback time series. This is 1X2877 matrix data has [a;b;c] for each
for each series? That makes no sense.
If you have 3 inputs the input matrix dimension should be [ 3 2877]!
Unless it is cell data and not matrix data...
To make sure, type the 4 commands
     iscell( [ JYTH ; JYE ] ) 
     [ I N ] = size(JYTH)
     [O N] = size(JYE) 
     whos
inputSeries = tonndata(JYTH,false,false); This is 1X2877 matrix data has target output targetSeries = tonndata(JYE,false,false);
You seem to be confused. tonndata produces cell data!
    whos inputSeries targetSeries
Create a Nonlinear Autoregressive Network with External Input inputDelays = 1:2; feedbackDelays = 1:2; hiddenLayerSize = 10;
How do you know these are good inputs? To be sure, calculate the significant lags for the output autocorrelation function AND the input/output crosscorrelation function
Choose Input and Feedback Pre/Post-Processing Functions... net.inputs{1}.processFcns = {'removeconstantrows','mapminmax'}; net.inputs{2}.processFcns = {'removeconstantrows','mapminmax'}; net.inputs{3}.processFcns = {'removeconstantrows','mapminmax'};
Delete these statements. These are defaults.
Prepare the Data for Training and Simulation [inputs,inputStates,layerStates,targets] = preparets(net,inputSeries,{},targetSeries);
Double check dimensions and data classes:
whos inputSeries targetSeries inputs inputStates layerStates targets
Setup Division of Data for Training, Validation, Testing The function DIVIDERAND randomly assigns target values to training, % validation and test sets during training...
 net.divideFcn = 'dividerand'; % Divide data randomly
NO, NO, NO!
RANDOM DIVISION DESTROYS AUTO AND CROSS CORRELATIONS
USE 'divideblock'
Hard to believe previous val and test results are "okay" if you used 'dividerand'
The property DIVIDEMODE set to TIMESTEP means that targets are divided into training, validation and test sets ... net.divideParam.trainRatio = 70/100; net.divideParam.valRatio = 15/100; net.divideParam.testRatio = 15/100;
These are defaults. Delete these statements unless you want to change the percentages.
Choose a Training Function ... net.trainFcn = 'trainlm'; % Levenberg-Marquardt Choose a Performance Function ... mse
These are defaults! Delete these statements unless you want to use other choices.
Choose Plot Functions... net.plotFcns = {'plotperform','plottrainstate','plotresponse', ... 'ploterrcorr', 'plotinerrcorr'};
These are part of a seven plot default list. Delete and you will get these PLUS ploterrhist and plot regression
Train the Network
[net,tr] = train(net,inputs,targets,inputStates,layerStates);
You can delete most of the the following statements. A detailed performance summary is already contained in the training structure tr. Type the command
tr = tr
Test the Network outputs = net(inputs,inputStates,layerStates); errors = gsubtract(targets,outputs); performance = perform(net,targets,outputs) Recalculate Training, Validation and Test Performance trainTargets = gmultiply(targets,tr.trainMask); valTargets = gmultiply(targets,tr.valMask); testTargets = gmultiply(targets,tr.testMask); trainPerformance = perform(net,trainTargets,outputs) valPerformance = perform(net,valTargets,outputs) testPerformance = perform(net,testTargets,outputs)
Most important are the test and validation results. If they are not very small compared to mean(var(testtarget',1)) and mean(var(valtarget',1)), respectively, then you need to improve your design by using a better choice of delays, or checking out other random weight initializations or maybe even changing the number of hidden nodes from H=10.
Next you should close the loop and test the closed loop trn, val and tst results.
Now is a reasonable time to try new data on the closed loopnet
View the Network view(net)
Probably should view the net right after the train statement
From this part I want to run a new test or forecast with new inputs This is a new inputs 1X960. The maxrix has the same structure for the testing [a;b;c]
 inputSeries2 = tonndata(AUGTH,false,false);
 [inputs2,inputStates2,layerStates2,targets2] = preparets(net,inputSeries2);
How can you generate layerstates2 and targets2 without a targetSeries2?
You have to use a closed loop net.
ALWAYS check the preparets I/O
 whos inputSeries2 inputs2 inputStates2 layerStates2 targets2
When I want to generate a new output from the network all"output2"(1X960) has NaN. I suspect that "inputStates2" has NaN value its second row. Would you let me know how I resolve the
See above
Hope this helps.
Thank you for formally accepting my answer
Greg
Related Question