MATLAB: Precisions obtained with network openloop NarX

Deep Learning Toolboxnarx

Write data from network openloop with NarX:

N=2200

Ndof=1449

Neq=2200

Ntrneq=1540

Nw=91

I=8

O=1

R2=0.9911

R2a=0.9907

R2trn=0.9914

R2tst=0.9900

R2val=0.9905

MSE=8.48e-5

MSEa=8.84e-5

MSEtrn=9.2e-5

MSEtst=9.2e-5

MSEval=9.065e-5

If you know someone tell me if there is any sign of overfitting appreciate it because I think it achieves high precision and the graph I get a little offset from the original. Variables used are highly correlated. We also appreciate and would tell me ontener several steps ahead with real data and because I get to perform one iteration morning values but need to continue iterating my future values of inputs and that is not possible. thank you very much

Best Answer

net = narxnet(ID,FD,H);

What are ID, FD and H?

Typically, the sign of overfitting is very high R2trn with much lower R2trna, R2val and R2tst.

You're looking good.

Greg

Related Solutions

MATLAB: As used in closed loop network NarX reality

I think the network is evaluated openloop and I think it gives good results,

What are I, N, ID, FD, H, Ntrn, Nval, Ntst, R2trn, R2trna, R2val and R2tst ?

but for now I do not believe them.

Why?
What do you get from the same data using closeloop?

My questions are as follows:

1 - I have evaluated the accuracy openloop, how would an in closeloop?. For a first simulation closeloop as inputs I have therefore created a matrix with the last two values InputSeries and say 30 inputs outside the training sample which would have an array of 2 * 32. To create the matrix newsTargets introduce targetSeries the last value of the array and fill with 31 NaN thereby get another matrix 1 * 32.

I don't understand where this data is coming from.

From here I simulate the network and gives me values that putting them together with real targets is a bit displaced. (I would like to send a screenshot of that graph, if you tell me in what way will send it). I think I perform the operation correctly, although I would like to evaluate the accuracy closeloop very strictly.

There is a way to see plots on ANSWERS. Fnd out how. Right now I need to see your code.

2 - My second question is because if I try the net for real, that is, suppose we are to March 4, 2013. I by a platform get data (RSI inputs and EMA), whereby I newInputs other matrix in which the last 2 introduce inputSeries values and RSI values and EMA today. (I think I'm doing well so far). To create another array newTarget do with the last value NaN targetSeries and 2 to have the same dimension as newInput (2 * 3) and newTarget (1 * 3) and performed the simulation to obtain the predicted value of the day March 5, 2013 . To continue iterating introduce the value of March 5 and get the value of the March 6, much like the network does NAR?, Is, in preparets command:

[inputs, inputStates, layerStates, targets] = preparets (net, inputSeries, targetSeries);?? and evaluate the network outside the sample.

Basically these are my doubts.

 I don't understand. Post the code.

Greg

MATLAB: Precision network analysis NarX

1. Use original trn/val/tst data and 'divideblock' to create and evaluate the open loop net.

2. If val and tst performance is unsatisfactory, try to design better nets via changing delays, numHidden, random initial weights and/or data-split ratios. Oherwise ...

3. Close the loop and evaluate using the ORIGINAL data. Unbiased val and tst performances are much more important than the biased trn performance (even if the latter bias is mitigated by a degree-of-freedom-adjustment).

4. Predict closed loop performance using NEW test input data. If corresponding NEW target data is available, evaluate the performance.

 I think data are insufficient to achieve the desired accuracy. Besides the variables are correlated but do not show much correlation (0.35 or so).

Are you referring to zero lag correlations obtained from corrcoef or dynamic correlations obtained from nncorr, xcorr or crosscorr? Nonzero lag correlations are usually significant above 0.14 or so.

 In openloop get accuracy R2a = 0.82 with MSE = 0.0016 ...

Relatively irrelevant. What are R2val and R2tst?

 and performing the analysis of R2 in training / validation / test none of them made me overfitting.

Overfitting is usually detected by high R2trn but low R2val, R2tst and possibly R2trna.

You think that you should use more data?

If Nw << Ntrneq, you probably have enough training data.

If std(MSEtst) << MSEtst you probably have enough test data and similarly for val.

Look up the expression for std(MSE) assuming MSE is Chi-square distributed in Wikipedia (I only remember it for binomially distributed classifications).

Hope this helps.

Thank you for formally accepting my answer.

Greg

Best Answer

Related Solutions

MATLAB: As used in closed loop network NarX reality

MATLAB: Precision network analysis NarX

Related Question