MATLAB: How to change learning rate and momentum in ‘trainlm’ neural network

trainlm

I have a feed forward neural network trained with trainlm function.Now i want to change the learning rate and momentum.Is there a specific default range for learning rate or Do I need to find the optimum range for learning rate?

Best Answer

use
options = trainingOptions('sgdm',...
    'MaxEpochs',60, ...
    'LearnRateDropFactor',0.1,...
    'LearnRateDropPeriod',1,...
    'LearnRateSchedule','piecewise',...
    'ValidationData',{valImages,valLabels},...
    'ValidationFrequency',50,...
    'Momentum',0.85,...
    'ValidationPatience',5,...
    'InitialLearnRate',0.1,...
    'MiniBatchSize',128,...
    'Verbose',true,...
    'executionenvironment','cpu',...
    'Shuffle','every-epoch',...
    'Plots','training-progress');
net = trainNetwork(...,...,...,options);
and choose the parameters accordingly.
For default values, see the docs of trainingOptions.
In terms of the learning rate and momentum, I typically start with a large one just to test the general behaviour and then I drop the learning rate to get more accurate results. Its difficult to give you some concrete advice because the optimal learning rate depends on your specific problem.
Related Question