Solved – Output of linear SVM model in Matlab using SVM-light

MATLABsvm

I am using SVM-light with Matlab, for linear SVM. I would like to understand the output model, but I cannot find any documentation or help about it.

Here is the output:

        sv_num: 639
   upper_bound: 547
             b: 1.4023
      totwords: 576
        totdoc: 2000
     loo_error: -1
    loo_recall: -1
 loo_precision: -1
      xa_error: 14.9500
     xa_recall: 82.6000
  xa_precision: 86.8559
       maxdiff: 9.9611e-004
    r_delta_sq: 575.0052
   r_delta_avg: 23.9792
  model_length: 0.4847
          loss: 357.5542
         vcdim: 136.1071
         alpha: [2000x1 double]
   lin_weights: []
         index: [2000x1 double]
        supvec: [639x576 double]
   kernel_parm: [1x1 struct]
example_length: 23.9792
             a: [2000x1 double]

So far, I understand there are sv_num=639 support vectors. However, there are 638 positive index in the index vector ; ditto for the positive values of the alpha vector. The first value of the alpha vector is zero, and the non-zero values are then between 2 and 639. Same remark about the supvec vector. So it seems there are indeed 638 support vectors.

Actually, I would like to get the hyperplane W and the parameter b myself to see if I could understand the output model correctly. Since the kernel is linear, I thought of a sum over some coefficients times some support vectors.

For information, a(i) "should" store the values alpha_i times y_i for training samples x_i. And there are 638 non-zero values.

Has anyone used svmlight before? Can you understand the output model? Or is there a documentation anywhere?

I manage to get one thing right:

MyCoeff = model.a ./ y;
MyIndex = 1+max(model.index,0); % between 1 and 639

The support vector corresponding to any non-zero MyCoeff(i) is:

  • x(i,:)
  • model.supvec(MyIndex(i),:)

And MyIndex is 1 iff MyCoeff is zero. And supvec(1,:) is the zero vector.

Here are four pieces of code I have made:

MyCoeff = model.a ./ y;
W1 = zeros(1, h*w);
for i=1:length(MyCoeff)
    W1 = W1 + MyCoeff(i)*x(i,:);
end;

W2 = zeros(1, h*w);
for i=1:model.sv_num
    W2 = W2 + model.alpha(i)*model.supvec(i,:);
end;

eq((MyCoeff~=0),(model.index>0))
MesIndices = 1+max(model.index,0); % entre 1 et 639
W3 = zeros(1, h*w);
for i=1:length(MyCoeff)
    W3 = W3 + MyCoeff(i)*model.supvec(MesIndices(i),:);
end;

W4 = zeros(1, h*w);
for i=1:length(MyCoeff)
    W4 = W4 +  model.alpha(MesIndices(i))*x(i,:);
end;

Here are the outputs:
http://imgur.com/FErWW.png

I have understood the index link between the support vectors supvec and the matrix x. But I have not found the link between alpha and a.

I got it: apart from the difference of indices, model.a and model.alpha are the same. 

for i=1:length(MesIndices)
    W1 = W1 + model.a(i)*x(i,:);%*y(i);
    W2 = W2 + model.alpha(MesIndices(i))*model.supvec(MesIndices(i),:);%*y(i);
    W3 = W3 + model.a(i)*model.supvec(MesIndices(i),:);%*y(i);
    W4 = W4 +  model.alpha(MesIndices(i))*x(i,:);%*y(i);
end;

The question now is whether I should multiply by y(i). I believe I should NOT since model.a is already y times the weight of the corresponding support vector, and $y\in\{-1,1\}$. So the good result should be the ORANGE face, which you can get by using one of the four lines above. Too bad I could not find any documentation. :')

Best Answer

sv_num: one added to the number of support vectors
     b: offset
 alpha: 0, then "a" corresponding to the support vectors, then only zeros
 index: link index between "a" and alpha, "x" and supvec 
supvec: 0, then "x" corresponding to the support vectors
     a: weights_i (between 0 and C) times y_i
Related Question