Solved – Extracting decision function variable from libsvm

kernel tricklibsvmsvm

I'm trying to use LIBSVM's single class SVMs for some classification and need to extract the following sum post classification (i.e. the variable that the decision function takes in)

$$
\Sigma_{i=1}^{N} \alpha_i K(z,z_i)
$$

I'm not too sure on this so I just want to check that I have the right idea, but could I calculate the sum just by using the primal variable $\textbf{w}$? So for a libsvm model $m$ I just call

w = (m.SVs)'*(m.coef);

and then calculate $\textbf{w}^T \textbf{z}$, where $\textbf{z}$ is the feature vector of what I'd like to classify. I'm not sure though, I guess it may just be easier calculating the kernel function explicitly, but if anyone knows of a good way to access the info in libsvm I would be very grateful

Best Answer

Depending on what kernel you use, you cannot possibly compute $\mathbf{w}$ (the separating hyperplane in feature space) explicitly. For the RBF kernel, for example, $\mathbf{w}$ is infinite dimensional.

Instead, what you can compute is the inner product of $\mathbf{w}$ and the test instance $\mathbf{z}$ embedded in feature space $\phi(\mathbf{z})$ as follows $\langle \mathbf{w}, \phi(\mathbf{z})\rangle = \sum_{i\in SV} \alpha_i y_i K(\mathbf{x}_i, \mathbf{z})$.

If your question is whether or not you can compute the decision value, you can do that as follows: $$f(\mathbf{z}) = \sum_{i\in SV} \alpha_i y_i K(\mathbf{x}_i, \mathbf{z}) +b.$$

In libsvm, the sv_coef variable contains the vector $\alpha_i y_i$.

Related Solutions

Solved – Plotting the decision boundary of a kernel SVM (RBF)

I figured out what is needed to be done. Actually, it's something simple, but its seems I had a matlaboid bug... Here is the code and the resulting figure for the "XOR" binary classification problem.

gamma     = getGamma();
b         = getB();
points_x1 = linspace(xLimits(1), xLimits(2), 100);
points_x2 = linspace(yLimits(1), yLimits(2), 100);
[X1, X2]  = meshgrid(points_x1, points_x2);

% Initialize f
f = ones(length(points_x1), length(points_x2))*rho;

% Iter. all SVs
for i=1:N_sv
    alpha_i = getAlpha(i);
    sv_i    = getSV(i);
    for j=1:length(points_x1)
        for k=1:length(points_x2)
            x = [points_x1(j);points_x2(k)];
            f(j,k) = f(j,k) + alpha_i*y_i*kernel_func(gamma, x, sv_i);
        end
    end    
end

surf(X1,X2,f);
shading interp;
lighting phong;
alpha(.6)

contourf(X1, X2, f, 1);

where the function

function k = kernel_func(gamma, x, x_i)
    k = exp(-gamma*norm(x - x_i)^2);
end

just produces the kernel function (RBF kernel), $k(\mathbf{x},\mathbf{x}')=\operatorname{exp}\left(-\gamma\lVert\mathbf{x}-\mathbf{x}'\rVert^2\right)$.

Here is the result for the XOR problem. Here $\gamma=4$.

enter image description here

Solved – One vs All appraoch to multiclassification in LIBSVM

well the ''-b'' 1 option that you have used on prediction gives you the probability of prediction (measure of confidence) . for 1 vs all classification you can use this link: http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#f808

Best Answer

Related Solutions

Solved – Plotting the decision boundary of a kernel SVM (RBF)

Solved – One vs All appraoch to multiclassification in LIBSVM

Related Question