MATLAB: Using different learning algorithms for the neural net toolkit

neuralneural networktraining

How do I go about implementing a genetic algorithm (for example) tool to minimise the weights for a neural network to find the global maxima? I'm worried that the built in trainers are not adequate to find the global minima.

Best Answer

You may be worrying about the wrong thing. With a typical I-H-O FFnet the number of equivalent nets obtained by just changing weight signs and index order is ~
 2^H * factorial(H) (= 3.7159e+09 for the 
 default H=10)
I find the best bet is to just find one of the many nets that MINIMIZE THE NUMBER OF HIDDEN NODES subject to the following maximum bound on mean-square-error.
   MSE = mse(error) <= 0.001*MSE00
where
error = target - output;
MSE00 = mean(var(target',1))  % Average target variance
The resulting bounds on normalized MSE and Rsquare (Google R squared) are
     NMSE = MSE/MSE00 <= 0.001
     Rsq  = 1 - NMSE  >= 0.999
which is interpreted as successfully modelling more than 99.9% of the target variance.
Initial weights are random. Therefore it is wise to make a double loop search over number of hidden nodes and initial random number states. I have posted zillions of examples in both the NEWSGROUP and ANSWERS. Good search words are
 greg Hmin Hmax Ntrials
If you insist on using a genetic algorithm see my post
http://www.mathworks.com/matlabcentral/newsreader/search_results?search_string=NEURAL+greg+genetic
Hope this helps.
Thank you for formally accepting my answer
Greg
Related Question