Solved – Feature selection using RFE in SVM kernel (other than linear eg rbf, poly etc)

feature selectionsvm

At this link, there is an example of finding feature ranking using RFE in SVM linear kernel.

If I want to check feature ranking in other SVM kernel (eg. rbf, poly etc).How to do it?

I have changed the kernel in the code from SVR(kernel="linear") to SVR(kernel="rbf"),

from sklearn.datasets import make_friedman1
from sklearn.feature_selection import RFE
from sklearn.svm import SVR
X, y = make_friedman1(n_samples=50, n_features=10, random_state=0)
estimator = SVR(kernel="linear")
selector = RFE(estimator, 5, step=1)
selector = selector.fit(X, y)
selector.ranking_

and then I get this error 

ValueError: coef_ is only available when using a linear kernel

Question: How to check feature ranking in other SVM kernels eg rbf, poly etc?

Best Answer

To use RFE, it is a must to have a supervised learning estimator which attribute coef_ is available, this is the case of the linear kernel. The error you are getting is because coef_ is not for SVM using kernels different from Linear. It is in RFE documentation

A walk-around solution is presented in Feature selection for support vector machines with RBF kernel by Quanzhong Liu et. al.

Related Question