MATLAB: Optimal hidden nodes number

feedforwardoptimal hidden nodestrial and error

Hello everyone, I would like to find optimal hidden nodes number using structured trial an error. I did the following simulation :
Hmin = 1;
Hmax = 30;
dH = 1; 
NTrials = 5;
I took the minimum error for each 5 trials to plot the following graph:
My question is how to determine optimal hidden nodes from this graph ? Thank you.

Best Answer

This is the approach I use. First I determine how many training equations are used
 Ntrneq = Ntrn*O  
 Ntrn   = number of training input/target pairs
 O      = dimension of the output target
Then, for a given MLP with an I-H-O network topology, the number of unknown weights is
 Nw = (I+1)*H+(H*1)*O.
 When
 H <= Hub = (Ntrneq-O)/(I+O+1),
the number of unknown weights does not exceed the number of training equations. Therefore, numerically estimated minimum error solutions are stable.
Otherwise Nw > Ntrneq and the net is OVERFIT. Then, unless precautions are taken, the dreaded phenomenon of
 OVERTRAINING an OVERFIT NETWORK
can occur and solutions can be useless.
Two ways to avoid this are
 1. Use a validation design subset that will cause training to be stopped 
    when the validation subset error increases continually for 6(MATLAB  
    DEFAULT) consecutive epochs.
 2. Bayesian Regularization via either
    a. The TRAINBR training algorithm
    b. Using TRAINLM (the default) with the error function MSEREG
My approach is to use a double loop solution to
 1. Minimize H
 2. Subject to the training target subset constraint
     mse(errortrn) <= 0.01 * mean(var(targettrn',1))
3. This results in a training subset Rsquare that is greater than 0.99 !!!
4 The outer loop is over h = Hmin:dH:Hmax
5. The inner loop is over Ntrials of random initial weight assignments.
Even though I always use a validation subset, I try 10 initial random weight trials for each trial value of H with H <= Hub. If unsuccessful, I then consider H > Hub.
I have posted hundreds of examples in the NEWSGROUP and ANSWERS. A good search term is
                       Hmin:dH:Hmax
Hope this helps.
Thank you for formally accepting my answer
Greg.
Related Question