MATLAB: How to determine feature importance of an SVM classifier

Question

I would like to calculate feature importance for a SVM classifier, e.g. by using the metric "mean decrease accuracy".

This means I need to know how the accuracy of my classifier (calculated by cross validation) changes if I leave out features one by one.

I found functions for classification trees, but nor for SVM. How could I calculate this for SVMs?

Answer 1

In general, unless you are using a linear kernel SVM, it is not possible to use the parameters of an SVM model to analyze the importance of your features. You can refer to the following external discussions for more information about this reasoning:

https://stackoverflow.com/questions/41592661/determining-the-most-contributing-features-for-svm-classifier-in-sklearn

https://stackoverflow.com/questions/21260691/scikits-learn-how-to-obtain-features-weight

Nevertheless, you can still analyze the feature importance for your classification problem (not specific to SVM) by doing some dimensional reduction or feature extraction.

For instance, you can perform neighborhood component analysis using the "fscnca" function in MATLAB to identify relevant features for your classification:

https://www.mathworks.com/help/stats/fscnca.html

Another popular technique for feature selection is sequential feature selection which can help you select features for classifying high dimensional data:

https://www.mathworks.com/help/stats/examples/selecting-features-for-classifying-high-dimensional-data.html

You can also refer to the following documentation link for other dimensionality reduction and feature extraction techniques in MATLAB:

https://www.mathworks.com/help/stats/dimensionality-reduction.html