Solved – SVM importance of predictor variables

feature selectionsvm

I am building a model in R using support vector machine (SVM) with KBF kernel.
The model seems to work quite well. I would like to assess the relative importance of predictor variables.

Can anyone tell me how this can done. Does there exist an R script for that?

Best Answer

Unfortunately, there is no direct way to get such information with SVMs, in contrast to, for example, logistic regression.

If you want, you can estimate the contribution of a single predictor ad hoc by training a model on all predictors except that specific predictor. The difference in performance between that model and the one with all predictors may then be considered the marginal contribution of that predictor.

If time permits, you can perform this procedure for all predictors.

Related Solutions

Solved – Output of linear SVM model in Matlab using SVM-light

sv_num: one added to the number of support vectors
     b: offset
 alpha: 0, then "a" corresponding to the support vectors, then only zeros
 index: link index between "a" and alpha, "x" and supvec 
supvec: 0, then "x" corresponding to the support vectors
     a: weights_i (between 0 and C) times y_i

Solved – Which kernel method gives the best probability outputs

Gaussian process classification (using Expectation Propagation) is probably the state-of-the-art in machine learning. There is an excellent book by Rasmussen and Williams (downloadable for free), the website for which has a very good MATLAB implementation. More software, books, papers etc. here. However, in practice, KLR will probably work just as well for most problems, the major difficulty is in selecting the kernel and regularisation parameters, which is probably best done by cross-validation, although leave-one-out cross-validation can be approximated very efficiently, see Cawley and Talbot (2008).

Best Answer

Related Solutions

Solved – Output of linear SVM model in Matlab using SVM-light

Solved – Which kernel method gives the best probability outputs

Related Question