MATLAB: How to change the activation function in ANN model created using toolbox

activation functionsDeep Learning ToolboxMATLABneural networks

I've created this model by editing the codes from the toolbox. The purpose of this model is to train the network with operating data from a turbine. the data is normalized and then the target will be set according to the actual fault occurrence which tagged as "1" and during normal operation "0". I will be comparing the result of several training function, the number of neuron, the number of layers, and activation function.
% This script assumes these variables are defined:
%   data - input data.
%   target - target data.
% load data
load data.mat;
load target.mat;
x = data;
t = target;
% Choose a Training Function
% For a list of all training functions type: help nntrain
% 'trainlm' is usually fastest.
% 'trainbr' takes longer but may be better for challenging problems.
% 'trainscg' uses less memory. NFTOOL falls back to this in low memory situations.
trainFcn = 'trainbr';  % Bayesian Regularization
% Create a Feedforward Network
hiddenLayerSize = 18;
net = feedforwardnet (hiddenLayerSize,trainFcn);
% Setup Division of Data for Training, Validation, Testing
RandStream.setGlobalStream(RandStream('mt19937ar','seed',1)); % to get constant result

net.divideFcn = 'divideblock'; % Divide targets into three sets using blocks of indices
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
%TRAINING PARAMETERS
net.trainParam.show=50;  %# of ephocs in display
net.trainParam.lr=0.05;  %learning rate
net.trainParam.epochs=10000;  %max epochs
net.trainParam.goal=0.05^2;  %training goal
net.performFcn='mse';  %Name of a network performance function %type help nnperformance
% Train the Network
[net,tr] = train(net,x,t); 
% Test the Network
y = net(x);
e = gsubtract(t,y);
performance = perform(net,t,y)
% View the Network
view(net)
The questions are: Is it correct to use this code below and will it affect the function of my model?
RandStream.setGlobalStream(RandStream('mt19937ar','seed',1)); % to get constant result
How to add another hidden layer? How to change the activation function for each layer? What is the best plot to show the capability of the ANN model to detect the fault of the turbine earlier than existing control system. Please advise me if there are anything that can be corrected/improved.

Best Answer

You are approaching the problem in exactly the wrong way.
The multilayer perceptron with one hidden layer is a universal approximator. The only reason to use more than one hidden layer is to reduce the total number of unknown weights by reducing the total number of hidden nodes (i.e., H1+H2 < H).
Ntrn training pairs of I-dimensional inputs and O-dimensional output targets yields Ntrneq = Ntrn*O training equations. The best way to obtain a robust design that tends to be resistant to noise, interference, measurement and transcription error is to MINIMIZE the number of unknown weights that yield an acceptable solution. If possible, Nw << Ntrneq is desirable.
 1. Use FITNET (calls FEEDFORWARDNET) for regression and curve-fitting
 2. Use PATTERNNET (calls FEEDFORWARDNET) for classification and pattern-recognition
 3. You have a classification problem. Start with the simple code in
     help patternnet
     doc patternnet
 4. If there are c classes, the target matrix columns should be columns of eye(c): O = c.
 5. The relationship between trueclass indices 1:c and the target columns is
 target           = ind2vec(trueclassindices);
 trueclassindices = vec2ind(target);
 6. Before starting the design, get a "feel" for the data. This may include
  a. plot inputs
  b. plot targets
  c. plot targets vs inputs
  d. standardize inputs to zero mean and unit variance using zscore or mapstd.
  e. Repeat a and c 
  f. Remove or modify errors and outliers.
 7. Start simple with the example used in the help and doc documentation.
  help patternnet
  doc patternnet
 8. You only have to vary 2 things 
  a. Number of hidden nodes (want as small as feasible) 
  b. Initial random weights
 9. This can be accomplished with a double for loop as I have illustrated in zillions of examples in the NEWSGROUP and ANSWERS. Search results
 NEWSGROUP HITS
 greg patternnet Ntrials  8
 ANSWERS HITS
 greg patternnet Ntrials           60
 greg patternnet Ntrials  Hmax     22
 greg patternnet Ntrials  Hub      17
 greg patternnet Ntrials  Hub Hmax 10
Hope this helps.
Thank you for formally accepting my answer
Greg
Related Question