MATLAB: What is the activation function of Levenberg–Marquardt Backpropagation algorithm

Deep Learning Toolboxlevenberg-marquardt algorithmMATLAB

Matlab has a Neural Network toolbox which helps to form a neural network automatically by training using Levenberg–Marquardt Backpropagation algorithm. I got the network and want to realize the hardware circuitry for this. I want to know the activation function used in this algorithm so that it would be easy for me to realise this network in hardware i.e in FPGA.

Best Answer

I don't think there is any limit on which differentiable performance functions are used except the output function must allow the range of normalized target values.

For FPGA I would recommend using rational approximations to the sigmoids.

 help/doc elliotsig
 help/doc elliot2sig

Or my preference

http://www.mathworks.com/matlabcentral/answers/56137-how-to-use-a-custom-transfer-function-in-neural-net-training

Hope this helps.

Thank you for formally accepting my answer

Greg

Related Solutions

MATLAB: How to train a feedfordward neural network with error weights

 Use the command window
 help mse
 doc mse

Hope this helps.

Thank you for formally accepting my answer

Greg

MATLAB: Mathe matical derivation for Levenberg–Marquardt algorithm

LM is not an equation solver. LM is a minimizer.

You can use it to minimize sum of squares of residues (so sum of squares of you equations in the situation where you are trying to find zeroes.) But if you are going to do that just for the sake of code generation, then you might as well just use fminsearch on the sum of squares of your equations instead.

LM gets stuck in local minima; fminsearch does as well, but fminsearch tends to get stuck less often than LM would.

Best Answer

Related Solutions

MATLAB: How to train a feedfordward neural network with error weights

MATLAB: Mathe matical derivation for Levenberg–Marquardt algorithm

Related Question