MATLAB: Is it necessary to have multiple filters for a convolution layer in a CNN

activationscnnfiltersnetneuralnnweights

I have a convolution layer in my CNN. For example, say the filter size is 2×2 and the stride is 2. There are 12 filters. What are the weights for the filters? How do they learn different features and activations? In order to get each filter to learn different weights, do I need to initialize the weights of each filter to different starting values?

Best Answer

The "trainNetwork" function in MATLAB will learn different weights for each filter, so that each filter learns different features. Although you can initialize each filter, this is not necessary as this will only be the starting point for the network training.

When you do transfer learning on a pre-trained network such as AlexNet, the initial weights before transfer learning are the weights that AlexNet learned during its training.

If you are training from scratch, you can initialize the weights to be random values. See the "Specify Initial Weight and Biases in Convolutional Layer" example at this link:

https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.convolution2dlayer.html

Related Solutions

MATLAB: Is fully connected layers quantization supported by Deep Network Quantizer

Answering the question myself: Yes, fc layers are currently not supported, and it is written at the top of the Deep Network Quantizer description: "Use the Deep Network Quantizer app to reduce the memory requirement of a deep neural network by quantizing weights, biases, and activations of convolution layers to 8-bit scaled integer data types."

But! You can replace fc layer with a conv layer with filters' size equal to prev layer output dims, so they will only be applied once.

MATLAB: Prediction on a pre-trained neural network

Using weights of different network to the current network:

for 2 networks net1 & net2, copy the layers to some variable

layers = net1.Layers;
layers(net1layerindex).Weights(net1index) = net2.Layers(net2layerindex).Weights(net2index);

Let's say you want to set weights of a channel of a convolution2dLayer to zeros(channelDimension)

layers(convLayerIndex).Weights(:,:,channelIndex) = zeros(channelDimension)
%also
layers(convLayerIndex).Weights(:,:,channelIndex,Index) = zeros(respectiveDimension)

In case of batchNormalizationLayer it has TrainedMean, TrainedVariance, Scale & Offset.

Use the above layers to create a DAG or a Series network. You can also refer to the documentation of the assembleNetwork function which helps creating deep learning networks from layers without training.

You can also set the Learning factor for a layer as follows:

layer.WeightLearnRateFactor = 0;

Best Answer

Related Solutions

MATLAB: Is fully connected layers quantization supported by Deep Network Quantizer

MATLAB: Prediction on a pre-trained neural network

Related Question