Solved – Number of kernel evaluations in SVM training

kernel tricksvm

What is the typical number of kernel evaluations (between two training vectors) performed during a (kernelized) Support Vector Machine (SVM) training?

I am asking this question because I need to determine how much I need to optimize the kernel in order to have some hope to do the training in a reasonable amount of time. The current kernel calculation time is prohibitive (1+ hour), but there is room for improvement (by many orders of magnitude).

Note: There are about 60k vectors in one class and 10k vectors in the other one.

Best Answer

That actually depends on the particular solver you are using, and the data you work with. It depends on the data, because the resulting number of support vectors you get depends on the problem at hand. As a consequence, adding/removing certain features might simplify your problem, thus taking less time to train and resulting in fewer support vectors.

In any case, this question is answered in detail in the paper Support Vector Machine Solvers, where a complexity of training a SVM is analyzed, and they also describe and compare different implementations.

Hope that helps.

ON REPLY TO YOUR FIRST REPLY: Do you use libsvm?. If so, you could try alpha seeding. LibSVM supports it. More details here and my reply to other question here for a quick intuitive description. Another approach migth be using stochastic gradient descent on SVM

Related Question