MATLAB: How do we decide the number of hiddenlayers in a PatternNet

classificationDeep Learning Toolboxhiddenlayersmachine learningMATLABneuralnetworknntoolboxpatternnet

Hi,

Well, since I'm working on PatternNets (Classification), my doubt is, how do we decide then number of hiddenlayers the network requires.

Before that, I have 8 predictors and 1 output response containing 4 classes.

I know that in order to define a hidden layer, we use :

MyNet = patternnet(10);

But how do I decide I need 10 and not 8 or any other number?

Any suggestion would be helpful. Thank you.

Best Answer

patternnet(10) indicates ONE HIDDEN LAYER WITH TEN NODES
It is important to be mindful of the number of layers and nodes.
The set of input nodes IS NOT CONSIDERED A LAYER ( Probably because no mathematical operations are performed there)
Three sets of nodes (INPUT/HIDDEN/OUTPUT) is sufficient. Let

size(inputmatrix) = [ I N ]

size(targetmatrix) = [ O N ]

Then with a single middle "HIDDEN" layer of size H,

The size of the typical two layer net is

size(net) = I - H - O

a. I input nodes

b. H hidden layer nodes

c. O output layer nodes

4. The total number of weights is

Nw = ( I + 1 ) * H + ( H + 1 ) * O

= O + ( I + O + 1 ) * H

where the "1s" represent constant bias connections

5. The data is typically divided into a training subset of size Ntrn ~ 0.7*N, a validation subset of size Nval ~ 0.15*N and a test subset of size Ntst ~ 0.15*N.

6. For good approximate solutions, the number of training equations Ntrneq = Ntrn*O should be sufficiently larger than the number of unknown weights Nw

7. The inequality Nw << Ntrneq results in

H << ( 0.7 * N * O - O )/(I+O+1)

8. A factor of 10 is usually sufficient

Hope this helps.

Greg

Related Solutions

MATLAB: Layer in neural network

Two of the most important uses for Neural Networks are regression(function approximation, curvefitting) and classification/pattern-recognition.

The best way to design NNs is via the Neural Network Toolbox

In general, the best function to use for regression is FITNET. Start with the help and doc documentation using the command line commands

 help fitnet
 doc fitnet

Next check out some of my posts on the NEWSGROUP and ANSWERS. Search using

 greg fitnet

In general, the best function to use for classification/pattern-recognition is PATTERNNET. Start with the help and doc documentation using the command line commands

 help patternnet
 doc patternnet

Next check out some of my posts on the NEWSGROUP and ANSWERS. Search using

 greg patternnet

Finally, use MATLAB data to practice with

 help nndatasets
 doc nndatasets

If you have problems, it is easier for us to help if you use those datasets.

The help and doc examples use as many default settings as possible. Therefore my best advice for beginners is to begin your designs similarly with one exception. Since by default, the trn/val/tst data division and initial weights are chosen randomly, it is important to know the initial state of the random number generator. The best way to do that is for you to set it to your favorite state before using TRAIN. If designing multiple designs in a loop, initialize RNG before the loop.

 help rng
 doc rng

The default topology is I-H-O ( H=10 default) for I dimensional input vectors and corresponding O-dimensional Output target vectors. The one hidden layer contains H=10 hidden nodes containing tansig (tanh) transfer functions.

The one hidden layer net is a universal approximator. Increasing the number of hidden nodes increases the ability to model more complex functions. However using more than needed decreases the ability to operate well on unseen data. The only reason to use more layers is if you want to decrease the total number of hidden nodes or, to try to model subclasses for classification.

 hiddenLayerSize=200

Replaces the 10 tansig hidden units with 200. Typically, this is a ridiculous number.

 net.divideParam.trainRatio=70/100 
 net.divideParam.valRatio=15/100 
 net.divideParam.testRatio=15/100

Divides the data into training, validation and test subsets with the ratio 70/15/15. However, this is the default ratio. Therefore those statements can be deleted.

Hope this helps.

Thank you for formally accepting my answer

Greg

MATLAB: How to choose number of hidden layers

When you call patternnet, pass it a vector of the sizes of the hidden layers.

https://www.mathworks.com/help/nnet/ref/patternnet.html

Best Answer

Related Solutions

MATLAB: Layer in neural network

MATLAB: How to choose number of hidden layers

Related Question