MATLAB: Do I get incorrect output when I use newff() function available in Neural Network Toolbox 6.0.2 (R2009a)

Deep Learning ToolboxincorrectNetworkneuralnewffoutputtoolbox

I am facing an issue using the NEWFF function when creating a feed-forward network using the LOGSIG functions for output neurons. The output of the network is in the range of [0.5 1] even though I expect it to be in the range of [0 1] for the target [0 1]. However this behavior is not observed by the use of a different function like TANSIG.
The code below provides you with the same.
2) Reproduction steps:
clear;
p = [-1 -1 2 2; 0 5 0 5];
t = [0 0 1 1];
% current interface
net = newff(p, t, 3, {'logsig','logsig'});
net.divideFcn = '';
net.trainParam.show = 50;
net.trainParam.lr = 0.05;
net.trainParam.epochs = 300;
net.trainParam.goal = 1e-5;
[net, tr, Y, E ] = train(net,p,t);
y = sim(net, p);
%%OUTPUT

% >> y

% y =

%    0.5000    0.5000    1.0000    1.0000
However, if I use output processing function MAPMINMAX the same code gives a different answer.
clear;
p = [-1 -1 2 2; 0 5 0 5];
t = [0 0 1 1];
minmaxP = minmax( p );
% minmaxP =
%    -1     2
%     0     5
numOutputs = size(t, 1);
% depreciated interface
net = newff(minmaxP, [3, numOutputs], {'logsig','logsig'});
net.divideFcn = '';
net.trainParam.show = 50;
net.trainParam.lr = 0.05;
net.trainParam.epochs = 300;
net.trainParam.goal = 1e-5;
[net, tr, Y, E ] = train(net,p,t);
y = sim(net, p);
%%OUTPUT
% >> y
% y =
%     0.0001    0.0001    0.9999    0.9946
Please explain the reason for difference in the output.

Best Answer

The reason that for this issue is that output processing function MAPMINMAX is mapping the target values into the standard range [-1, 1] used by the default TANSIG transfer function. Since the LOGSIG transfer function can only output [0,1] it is only actually able to fit pre-preprocessed targets in the range [0.5 1].
The solution is to ensure MAPMINMAX maps targets to [0, 1], to match the output range of LOGSIG:
p = [-1 -1 2 2; 0 5 0 5];
t = [0 0 1 1];
net = newff(p, t, 3, {'logsig','logsig'});
% *** Map targets to LOGSIG output range [0, 1]. ***
net.outputs{2}.processParams{2}.ymin = 0;
net.divideFcn = '';
net.trainParam.show = 50;
net.trainParam.lr = 0.05;
net.trainParam.epochs = 300;
net.trainParam.goal = 1e-5;
[net, tr, Y, E ] = train(net,p,t);
In general using the default TANSIG transfer functions, output processing function MAPMINMAX ensures that no matter what range the data has, it will be properly mapped for a TANSIG output layer.
Related Question