MATLAB: How to use the Genetic Algorithm (GA) to train a Neural Network in Neural Network Toolbox

algorithmgageneticGlobal Optimization ToolboxNetworkneuraloptimizetrain

How can I use the Global Optimization Toolbox "ga" to optimize the Neural Network?

Best Answer

The ability to set the algorithm to ga in the train function is not currently directly available in Neural Network Toolbox (as of R2017a at least).
To work around this issue, use the steps outlined below to optimize a neural network using a genetic algorithm.
The "ga" function requires a function handle as an input argument to which it passes a 1xN vector, where N is the number of variables in the system to be optimized.
For a neural network, the weights and biases are a Mx1 vector. These may be optimized using "ga".
A function can be written to accept the network, weights and biases, inputs and targets. This function may return the mean squared error based on the outputs and the targets as "ga" requires a function handle that only returns a scalar value.
The following code example describes a function that returns the mean squared error for a given input of weights and biases, a network, its inputs and targets.
function mse_calc = mse_test(x, net, inputs, targets)
    % 'x' contains the weights and biases vector
    % in row vector form as passed to it by the
    % genetic algorithm. This must be transposed
    % when being set as the weights and biases
    % vector for the network.
    %











    % To set the weights and biases vector to the
    % one given as input
    net = setwb(net, x');
    %
    % To evaluate the ouputs based on the given
    % weights and biases vector
    y = net(inputs);
    %
    % Calculating the mean squared error
    mse_calc = sum((y-targets).^2)/length(y);
end
The following code example describes a separate script that sets up a basic Neural Network problem and the definition of a function handle to be passed to "ga". It uses the above function to calculate the Mean Squared Error.\n
% INITIALIZE THE NEURAL NETWORK PROBLEM %
%
% inputs for the neural net
inputs = (1:10);
% targets for the neural net
targets = cos(inputs.^2);
%
% number of neurons
n = 2;
%
% create a neural network
net = feedforwardnet(n);
%
% configure the neural network for this dataset
net = configure(net, inputs, targets);
%
% create handle to the MSE_TEST function, that
% calculates MSE
h = @(x) mse_test(x, net, inputs, targets);
%
% Setting the Genetic Algorithms tolerance for
% minimum change in fitness function before
% terminating algorithm to 1e-8 and displaying
% each iteration's results.
ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');
%
% PLEASE NOTE: For a feed-forward network
% with n neurons, 3n+1 quantities are required
% in the weights and biases column vector.
%
% a. n for the input weights
% b. n for the input biases
% c. n for the output weights
% d. 1 for the output bias
%
% running the genetic algorithm with desired options
[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);
Please note that the above example makes use of
feedforwardnet
, which was first introduced in R2010b.
Related Question