MATLAB: What is the search range of a hidden layer size in a regression task

neural network

Searching for answers here in the forum, I read one answer by Greg that usually do:
"An outer loop H = Hmin:dH:Hmax over number of hidden nodes and an inner loop i = 1: Ntrials over number of random trn/val/tst data divisions and random weight initialization trials for each value of H."
What should be Hmin and Hmax ? I tried 0 and 20, it founds an answer at 20. So I grow the range to 30, and found an answer at 30. The MSEs between the first answer and the second results are similar (but the second has lower mse). Should I grow to 40 ?
When to stop ?

Best Answer

[ I N ] = size(x)
[ O N ] = size(t)
Ntrn = N - 2*round(0.15)*N) % default
Ntrneq = Ntrn*O % No of training equations
% No. of unknown weights for an I-H-O node topology
Nw = (I+1)*H+(H+1)*O = (I+O+1)*H + O
% To prevent more unknowns than equations
Nw <= Ntrneq <=> H <= Hub % upper bound
Hub = floor( ( Ntrneq-O) / ( I + O + 1) )
% Depending on your prior information and the size of Hub,
% choose Hmin, dH and Hmax to search
h = Hmin:dH:Hmax % 0 <= Hmin <= Hmax <= Hub
For each h candidate, design multiple nets. To keep the task manageable, I usually search 10 h candidates at a time with Ntrials = 10 designs per candidate to obtain 100 designs. Sometimes it may be necessary to start with a wide search followed by one or more narrow searches.
Don't forget to choose a repeatable initial random number seed so that you can reproduce any individual design.
Choose the smallest H that satisfies your training goal. For example: if the degree-of-freedom adjusted training Rsquare, R2trna, is greater than 0.995 then 99.5% of the training target variance is modeled by the net.
I have posted zillions of examples in the NEWSGROUP and ANSWERS. Search with one or more of
greg neural h = Hmin:dH:Hmax Ntrials R2trna
Hope this helps.
Thank you for formally accepting my answer
Greg