Solved – SVM in R package e1071

rsvm

I am trying to use SVM to make a prediction (True or False) on a dataset with many independent variables. I am wondering how I can identify the most useful variable in making the prediction. I understand that there is a function that allows me to calculate the "weights" of the variables in R package e1071.

w=t(model$coefs)%*% model$SV

But I am not sure how I can decide which are the important variables, based on these weights.

Best Answer

The caret package has useful utilities for variable selection. The rfe function performs recursive feature selection. Reference this answer for a demo: https://stackoverflow.com/a/17530483/819544

Related Solutions

Solved – How to obtain decision boundaries from linear SVM in R

For data point $x$ your SVM calculates decision value $d$ in the following way:

d <- sum(w * x) + b

If $d > 0$ then label of $x$ is $+1$, else it's $-1$. You can also get labels or decision values for data matrix newdata by saying

predict(m, newdata)

predict(m, newdata, decision.values = TRUE)

Be cautious when using SVM from package e1071, see Problem with e1071 libsvm? question. Several other SVM packages for R are kernlab, klaR and svmpath, see this overview: Support Vector Machines in R by A. Karatzoglou and D. Meyer.

Solved – How does SVM prediction work

For what I understand, once a new document arrives, SVM just applies the discriminant function and decides if the document is going to be classified or not. This would mean that the kernel function is not exploited in the test phase.

You can find a very good introduction to SVM in this tutorial by a UCL PhD student. In addition, this videolecture from the Machine Learning Summer School (from 2006) is very informative as well.

Regards,

Best Answer

Related Solutions

Solved – How to obtain decision boundaries from linear SVM in R

Solved – How does SVM prediction work

Related Question