MATLAB: Low performance of neural network using logsig for output layer

classificationDeep Learning Toolboxneural networkStatistics and Machine Learning Toolbox

Hi.

I have a binary classification problem and using “newff” for it. There is a single output in my neural network structure. If this outputs is equal or greater than 0.5 it belongs to class 1 and if smaller than 0.5 it belongs to class 0. So my targets for every sample is 0 or 1. Besides it I am normalizing data with “Mapstd” or “mapmaxmin”.

When I use “tansig” transfer function for hidden layer(s) and “purelin” for output, classification accuracy of network is good but when I change “purelin” to “logsig” the classification accuracy is really bad (0.5) and the classification accuracy is 50% in all repeats. What is the problem? (I'm using "Trainlm" without any normalization for outputs)

PS.

When I checked outputs after training, many of them are greater than 0.5 .

Thanks.

Best Answer

newpr is the version of newff that is to be used for classification; newfit is for regression.

All three are obsolete (see the help and doc documentation).

Currently, patternnet is the version of feedforwardnet that is to be used for classification; fitnet is for regression.

The default output transfer function is determined according to the default output normalization function.

Therefore, for each of the two groups of 3, it is worthwhile for you to go to the help, doc and type examples in the documentation and compare the properties of the three. For the current three

 default output normalization is mapminmax [-1,1]
 default output transfer functions are tansig for patternnet and purelin for feedforwardnet and fitnet
 Therefore, changing the output function of feedforwardnet from purelin to logsig will not 
yield the correct results because the range of logsig is not {-1,1}.
 If you want to use logsig, normalized targets must be in {0,1}; The best way to do this is just to use mapminmax BEFORE train to convert targets to { 0,1 }. Then disable mapminmax during training.
 This may also be a way of using softmax. However, I'm not familiar with MATLAB's version.

Hope this helps.

Thank you for formally accepting my answer

Greg

Related Solutions

MATLAB: Neural Network Plotting Confusion Matrix

Click on the plotconfusion button in the nntraintool window.

You have specified many net properties that are already defaults

... a waste of time and space.

Thank you for formally accepting my answer

Greg

MATLAB: How can i changhe the transfer function of output layer of neural network

With obsolete (since 2010b) nets like newfit for regression/curve-fitting and newpr for classification/pattern-recognition (both call newff), you can specify the transfer functions in the net creation statement. See

   help newfit , help newpr

However, in their replacements fitnet and patternnet (both call feedforwardnet), you have to specify them as you have indicated. The default configurations are 2 layer nets with layer 2 containing the output transfer function.

I do not recommend using non-differentiable functions like hardlim and hardlims.

For best performance use scaled centered inputs via mapminmax, mapstd or zscore. The corresponding compatible hidden layer transfer function is the symmetric TANSIG (i.e., I do not recommend LOGSIG for hidden layers).

For fitnet, also use scaled centered outputs with PURELIN or TANSIG as output transfer functions.

For patternnet, outputs are desired to be consistent estimates of the input conditional class posterior probabilities. Therefore targets should be unit column vectors with the "1" in the row corresponding to the true class of the corresponding input. The corresponding output transfer function is LOGSIG.

   help ind2vec , help vec2ind
   net.layers{2}.transferFcn = 'logsig'

Hope this helps.

Thank you for formally accepting my answer

Greg

Best Answer

Related Solutions

MATLAB: Neural Network Plotting Confusion Matrix

MATLAB: How can i changhe the transfer function of output layer of neural network

Related Question