MATLAB: Do I get different results for a neural network when I explicitly specify the number of inputs and input size

accuracyaccuratedifferentMATLABNetworkneuralresultsshallow

Assume that I create two neural networks, each having the same structure (for example, one, 10 neuron hidden layer) and a dataset 'data.mat', which has the variables 'x' and 't' corresponding to 'features' and 'targets' respectively.
The only difference between the two networks is that the first network has explicitly specified number and size of inputs while the second network has not.
For a concrete example, consider the following MATLAB script:
% load data
load('data.mat');
trainFcn = 'traincgf';
n=10;
% create and configure net1
net1 = fitnet(n,trainFcn);
net1 = configure(net1,'output', t);
% specify inputs and connections manually
net1.numInputs = 1;
net1.inputs{1}.size = 20;
net1.biasConnect =[ 1;1 ];
net1.inputConnect = [1;0];
net1.layerConnect = [0 0;1 0];
net1.outputConnect = [0 1];
% specify data split
net1.divideFcn = 'divideind';
[trainInd,valInd,testInd] = divideint(50, 0.7, 0.15, 0.15);
net1.divideParam.trainInd = trainInd;
net1.divideParam.valInd = valInd;
net1.divideParam.testInd = testInd;
% train net1
[net1,tr1]= train(net1,x,t);%,'CheckpointFile','ProteomicYield1000.mat');
y1=net1(x);
% create and configure net2
net2 = fitnet(n, trainFcn);
net2 = configure(net2,'output',t);
% Don't specify inputs or connections manually
% use the same data split as net1
net2.divideFcn = 'divideind';
net2.divideParam.trainInd = trainInd;
net2.divideParam.valInd = valInd;
net2.divideParam.testInd = testInd;
% train net2
[net2,tr2]= train(net2,x,t);
y2=net2(x);
Here 'net1' and 'net2' are two identical shallow neural networks. In 'net1', I explicitly specify the number and size of inputs after configuring the network.
In 'net2', I do not specify the number and size of inputs and directly configure and train the network.
I am running the above code on the same data, ensuring that the training, validation and test sets are exactly the same for both of the networks.
Why do I get very different results for 'net1' and 'net2', and 'net2' always seems to be more accurate than 'net1'?

Best Answer

The weights for the first network, 'net1', are not getting initialized properly. Specifically, 'net1' has two sets of weights, stored in variables 'IW' [input weights] and 'LW' [layer weights]. When you call the 'configure' function as follows:
net1 = configure(net1, 'output', t);
the layer weights in 'LW' get initialized because the network has enough information to know the size of these weights. However, the input weights in 'IW' do not get initialized because MATLAB doesn't know the input size yet. This is expected behavior.
Subsequently, when you set the size of the inputs, as follows:
net1.inputs{1}.size = 240;
MATLAB now has enough information to initialize the input weights 'IW'. However, the network does not have any input data available, so 'IW' is initialized to 'zeros' by default.
Initializing the weight matrix of the neural network to zeros is the root cause behind the poor accuracy of the network in the first script.
The are two ways to fix this issue:
1) You can re-configure the network using the inputs before training, as follows:
net1 = configure(net1, x, t);
This will initialize the input weights correctly.
2) Alternatively, you can delete the following line:
net1.inputs{1}.size = 240;
This will leave the input weights 'IW' un-initialized and calling the 'train' function will initialize 'IW' to appropriate values.
Now, 'net1' and 'net2' should perform similarly as both of them have been assigned similar initial weights.
Related Question