SVM – Variable Importance in Machine Learning Models
feature selectionmachine learningsvm
How to obtain a variable (attribute) importance using SVM?
Best Answer
If you use l-1 penalty on the weight vector, it does automatic feature selection as the weights corresponding to irrelevant attributes are automatically set to zero. See this paper. The (absolute) magnitude of each non-zero weights can give an idea about the importance of the corresponding attribute.
Also look at this paper which uses criteria derived from SVMs to guide the attribute selection.
Unfortunately, there is no direct way to get such information with SVMs, in contrast to, for example, logistic regression.
If you want, you can estimate the contribution of a single predictor ad hoc by training a model on all predictors except that specific predictor. The difference in performance between that model and the one with all predictors may then be considered the marginal contribution of that predictor.
If time permits, you can perform this procedure for all predictors.
You might take a look at this blog post on variable importance for neural network which also gives you ideas for graphical representation of NN with VI. Also see this Cross Validated question on VI for SVM and answers therein. You could calculate your VI for each of your set of models and take a look at the set of VIs across the board.
Best Answer
If you use l-1 penalty on the weight vector, it does automatic feature selection as the weights corresponding to irrelevant attributes are automatically set to zero. See this paper. The (absolute) magnitude of each non-zero weights can give an idea about the importance of the corresponding attribute.
Also look at this paper which uses criteria derived from SVMs to guide the attribute selection.