Solved – Training a Support Vector Machine in Matlab – number iterations exceeded

MATLABsvm

I'm trying to train a support vector machine on a set of training data (which is seperate to my testing data).

training_data has size MxN where M = 10175 and N = 102

Each observation has a label 1 or 0.
(118 observations have label 1, 10057 observations have label 0).

Initially, I ran the Matlab Statistics Toolbox method svmtrain like so:

svmtrain(training_data,labels,'Kernel_Function','rbf');

These are the default parameters recommended by Matlab.

However after a while I get this message

---------------------------
Error using seqminopt>seqminoptImpl (line 198)
No convergence achieved within maximum number of iterations.

Error in seqminopt (line 81)
[alphas offset] = seqminoptImpl(data, targetLabels, ...

Error in svmtrain (line 499)
[alpha, bias] = seqminopt(training, groupIndex, ...

What next steps should I take?

I tried using the 'Kernel_Function','linear' option but the same problem happened.

I'm not sure whether If I should use my own values for rbf_sigma or even for boxconstraint? If anyone can help guide me in getting a SVM to be trained then that would be really helpful!

PS. Could there be something about my data that means the svmtrain function will always exceed the maximum number of iterations, i.e. never converge?

I don't want to aimlessly try different combos svmtrain method parameters, there are simply too many!

Best Answer

I would recommend using LibSVM which you can call from Matlab, it is generally much faster and better. As for the parameters: you need to optimize both of these by splitting your data in three pieces and testing different values on a part of the data. Basically you would procede as follows (fractions are just a choice):

  • Split off 1/3 training data
  • Split off 1/3 validation data
  • Split off 1/3 testing data

You would train the SVM using a set of parameters on the training data and test it on the validation data and repeat this procedure for different parameters. Once an optimal parameter set is found, you then train on both the training and validation data (put them together) and evaluate your model on the testing data.

As for the parameter testing, this is usually done in a 'grid search' with a procedure called 'cross-validation' (see also the part about k-fold cross-validation here).

One more thing: if you were to use a linear kernel, things would probably go much faster. The kernel itself is much faster to evaluate and you would only need to optimize one parameter (the box constraint) instead of two (the box constraint and the sigma bandwidth). That being said, I have worked with dimensions much larger than yours in a matter of seconds using LibSVM.

All of this and more is als explained in 'A Practical Guide to Support Vector Classication', which I strongly recommend you read before starting to use SVMs.

Related Question