MATLAB: Can we only train the classification layer when do transfer learning of a pre-trained network

Deep Learning Toolboxtransfer learning deep neural network

Hi,

My question relates to this article https://uk.mathworks.com/help/deeplearning/gs/get-started-with-transfer-learning.html

The question is, can we only train the classification layer when do transfer learning of a pre-trained network? I want to speed up my training by keeping the feature extraction layers (base model) as they are and only replace and retrain the classification layers.

The equivalent way in Keras (Python) is by: base_model.trainable = False

If possible in Matlab, please let me know how. Your help is appreciated.

Cheers

Sud

Best Answer

In order to freeze the weights of a particular layer of your network set the properties WeightLearnRateFactor & BiasLearnRateFactor to zero. Refer to fullyconnectedLayer - Learn Rate and Regularization, convolution2dLayer - Learn Rate and Regularization & lstmLayer - Learn Rate and Regularization.

layer.WeightLearnRateFactor = 0;

You can also refer to Freeze Initial Layers of the Train Deep Learning Network to Classify New Images example.

Related Solutions

MATLAB: Transfer learning layers in alexnet and googlenet

Here is a page where you can compare pretrained networks: https://www.mathworks.com/help/deeplearning/ug/pretrained-convolutional-neural-networks.html

MATLAB: How to freeze specific weights of Neural network model

Hi Abdelwahab,

If you wish to assign those weights in the beginning and keep them as constant, you can set the ‘WeightLearnRateFactor’ property as 0 (which defines the learning rate for that layer) for those layers.

For example,

convolution2dLayer(3,1,'Padding',[1 1 1 1],'WeightLearnRateFactor',0);

You may refer to this link to learn more about such properties.

Or if you want to fix certain weights to some layers in a trained network , then directly assign those layers the values after training the network.

 net = alexnet; % or your pre-trained network 
 layer = net.Layers(1) % here 1 can be replaced with the layer number you wish to change 
 layer.Weights = randn(11,11,3,96); %the weight matrix which you wish to assign

Best Answer

Related Solutions

MATLAB: Transfer learning layers in alexnet and googlenet

MATLAB: How to freeze specific weights of Neural network model

Related Question