MATLAB: Neural network accuracy improves on retraining without weight reinitialisation

initialisationneural networknftoolnntooltraining

Apologies as a similar question has been asked before, but it was never resolved. I am trying to create a neural network for use in a regression problem using nftool/nntool. I find that, on the first training run, the network sometimes performs quite poorly, but that with subsequent training runs regression accuracy seems to increase (pretty much with each successive run), although the weights have not been reinitialised. Why does this happen (answers in terms of the error surface and backpropogation would be illustrative though I don't need that much detail)? When the weights are not reinitialised, does each training run in MATLAB somehow 'build on' the previous?

Thanks

Best Answer

A net with former weights will continue training from those weights.

If you wish to reinitialize to get an alternate design use the function configure at the top of the loop.

I have posted zillions of examples in both the NEWSGROUP (now only available in comp.soft-sys.matlab) and ANSWERS

Try searching

 greg configure

Hope this helps.

Thank you for formally accepting my answer

Greg

Related Solutions

MATLAB: How to train a feedfordward neural network with error weights

 Use the command window
 help mse
 doc mse

Hope this helps.

Thank you for formally accepting my answer

Greg

MATLAB: Recognize overfitting in retraining

Recognize overfitting in retraining Asked by Federico Ambrogio on 25 Sep 2015 at 9:47 I wrote the following code, inspired of those proposed in the neural network toolbox manual, to retrain a network load dati_MRTA.mat% where IN_MRTA=13x49 double and TARGET_MRTA=1x49 double

1. OVERFITTING and OVERTRAINING

When

 [ I N ] = size(input)  % [ 13 49 ]
 [ O N ] = size(target) % [  1 49 ]
 and
 H is the number of hidden nodes  % 13
 Nw = (I+1)*H+(H+1)*O = 14*13+14*1 = 196

is the number of unknown weights that have to be estimated. OVERFITTING occurs when there are more unknown weights than there are training equations

 Nw =196 > 49 = N*O >= Ntrn*O = Ntrneq

Problems occur when you OVERTRAIN an OVERFIT net. Three methods of preventing overtraining are

 a. Do not overfit H <= Hub (upperbound) where,
    using the MATLAB default Ntrn ~ 0.7*N
    Hub = floor(( Ntrn*O-O)/(I+O+1))
       = floor(( 0.7*N*O-O)/(I+O+1) ) = 2
 b. Validation Stopping
 c. Regularization using MSEREG or TRAINBR.
 2. Train automatically divides the data into three
subsets trn/val/tst with ratios 0.7/0.15/0.15.

Therefore, either accept this or EXPLICITLY override it with net.divideParam.trainRatio, etc. to get the three unit-sum ratios. There is no need for you to explicitly divide the data !

3. When training in a loop, the net must be reconfigured with function CONFIGURE at the beginning of each pass through the loop. Otherwise you will just continue to train the same net with initial weights equal to the final weights obtained from the previous pass.

4. Initialize the RNG before using the 1st random number so you can reproduce your results that depend on both random data division and random weight intialization.

5. Use the regression function fitnet instead of the general function feedforwardnet.

6. You can obtain zillions of examples by searching both the NEWSGROUP and ANSWERS using the search words

greg fitnet tutorial
greg fitnet Hub
greg fitnet Ntrials
 Hope this helps.

Thank you for formally accepting my answer

 Greg

Best Answer

Related Solutions

MATLAB: How to train a feedfordward neural network with error weights

MATLAB: Recognize overfitting in retraining

Related Question