Solved – Positive or negative effect of neural network inputs on output in binary classification (MATLAB)

importanceMATLABneural networkssensitivity analysis

How we can find an input has positive or negative effects on output in a binary classification neural network in MATLAB R2015a? (with softmax transfer function in output layer, transig for two hidden layers, 5 neurons in first hidden layer and 7 neurons in second hidden layer + 10 inputs).
How we can know which inputs have higher weights on output in a trained neural network (MLP)? (most impotent inputs – ranking)?

Best Answer

The most common approach to find feature importance, is to employ a generalized linear model and check its performance by turning off features. The method is described here:

http://uk.mathworks.com/help/stats/feature-selection.html?refresh=true.

Another approach, that I usually prefer, is to use random forests to compute the feature importance based on their splits and OOB samples. You can find more information at:

http://uk.mathworks.com/help/stats/ensemble-methods.html#bsx62vu, and in order to calculate the variance importance, you just set the 'OOBVarImp' as 'On' in the treebagger function

If you still want to use a neural network, and given that your features are standardized, the method that you have to follow is called Sensitivity Analysis (as you have it in your tags). It does exactly what you want, but probably you will have to code it yourself. I would refer you for more information about implementing it to:

Related Solutions

Solved – Neural networks creates negative output

One reason that this could happen is that the network hasn't converged.

Another reason could be that this particular network is not the best architecture to solve this problem.

But a direct solution to this problem would be to use an activation function in the final layer that is restricted to be in [0,1] such as the logistic function -- this will never produce negative outputs. Moreover, since you've scaled your target to be in $[0,1]$ as well, the predictions will always be in the same interval as the target.

Solved – Find Neural Network Inputs Given Outputs

I'm no expert in this field, so I might be wrong. Therefore, correct me if I'm wrong.

consider this neural network (which I suppose is equivalent to yours):

A---H1
 \ /  \
  X    C
 / \  /
B---H2

consider that the activation function of H1, H2 and C is the bipolar sigmoid, to which we'll refer to as "bsig(x)"

also, we'll name the connections as follows:
A, H1: wa1;
A, H2: wa2;
B, H1: wb1;
B, H2: wb2;
H1, C: wh1;
H2, C: wh2

now the values of H1, H2 and C can be defined as:

H1 = bsig(wa1 * A + wb1 * B)
H2 = bsig(wa2 * A + wb2 * B)
C = bsig(wh1 * H1 + wh2 * H2)

So, C can be written as:

C = bsig(wh1 * bsig(wa1 * A + wb1 * B) + wh2 * bsig(wa2 * A + wb2 * B))

All you need to do is solve this equation in order to B or A depending on which of the values is unkown.

Best Answer

Related Solutions

Solved – Neural networks creates negative output

Solved – Find Neural Network Inputs Given Outputs

Related Question