MATLAB: How do i calculate the accuracy of ANN

neural networktrainlm

how do i calculate the accuracy of ANN?

i have 10 structures of neural network. i want to find the best structure that can predict as close as possible to the actual value.i use trainlm. so, my question is,how can i choose the best structures other than mean absolute percentage error (MAPE)?

Best Answer

So this is regression, not classification. Use fitnet and vary the number of hidden nodes. For each value of hidden nodes, design Ntrials nets that differ by the set of random initial weights.

For examples, search the NEWSGROUP and ANSWERS using

 greg fitnet Hub Ntrials

Use the training record tr to separate the trn/val/tst indices and performances.

Choose a performance index

MSE % scale dependent Useless unless compared with something

NMSE = MSE/mean(var(target',1)) % Normalized MSE; Typically 0 <= NMSE <= 1

R2 = 1-NMSE % fraction of target variance that is modeled by the net (Search Rsquare in Wikipedia)

Separately tabulate the trn/val/tst results. Choose the net that has the best validation performance. If there are several with performances that are not significantly different, choose one with the smallest number of hidden nodes.

Use the corresponding test performance to predict performance on unseen data if the net is deployed.

Searching the NEWSGROUP and ANSWERS with

 greg ntrials

will provide examples.

Thank you for formally accepting my answer.

Greg

Related Solutions

MATLAB: Normalize Inputs and Targets of neural network

MAPMINMAX can be inappropriate only if there are outliers.

Use MAPSTD to detect outliers that can be removed or modified.

Hope this helps.

Thank you for formally accepting my answer

Greg

MATLAB: How it is possible to have overfitting before the network learn properly

Poorly worded question

Are we supposed to guess

 1. That you are referring to a classifier ?
 2. Which MATLAB function you are using ... patternnet ?
 3. The number of classes c ?
 4. The dimensionality of the inputs I ?
 5. The number of hidden nodes H ?
 6. The trn/val/tst ratio  0.7/0.15/0.15 ?

Overfitting only means that you have more unknown weights than training equations

 Nw > Ntrneq
 where
 Ntrneq = Ntrn*c
 Nw = (I+1)*H+(H+1)*c

Validation stopping has nothing to do with training data performance. It has to do with

 OVERTRAINING AN OVERFIT NET

It means that the training has reached the point where validation set performance (mse or cross-entropy) has undergone a local minimum indicating that if you don't stop, you will have over-trained an over-fit net to the point where further training will probably make the net perform worse on val, tst and unseen non-training data.

Remember:

The goal of design is to use training data to obtain a net that works well on all nontraining data:

 validation + test + unseen

Hope this helps.

Thank you for formerly accepting my answer

Greg

Best Answer

Related Solutions

MATLAB: Normalize Inputs and Targets of neural network

MATLAB: How it is possible to have overfitting before the network learn properly

Related Question