According to the book "Support Vector Machines" from Cristianini and Shawe-Taylor, it is feasible to make kernels from kernels.
My question is now more in application of this methods with tools like R.
In particular, I have two models with the same classes, but the data is different.
Do you know how I can combine two SVM kernels, i.e. linear and radial, to one learner for the classification task?
Can you provide me some useful documents or links to libraries to do so?
Best Answer
As Dougal comments, you can just add, average or multiply the values of individual kernels. By way of example, the authors of this paper (1) propose methods for combining kernels in the context of ordinal, categorical and continuous-valued data found in clinical medicine. Their solution involves computing a kernel function for each feature, with the function to be computed depending on the type of data involved: they have an ordinal kernel, a categorical kernel and a continuous-valued kernel. Then they scale each of the function values to the $[0,1]$ interval and average the several kernel matrices to produce their final kernel matrix.
To your specific question about how to combine radial and linear kernels, you should consider what properties you want the kernel function to have. The only constraint on kernel functions is that they must be symmetric, positive semi-definite, but within those very general constraints, you can make manipulations of the radial and linear kernel functions' values to emphasize some attributes of your data, or de-emphasize others.
As for achieving a combination of kernel functions with software, that's a programming problem, rather than a statistical one... But in R, supposing that you want to average two kernel matrices $A$ and $B$ of the same dimension, you can use something like
and the result is also a square kernel matrix of the same dimension as $A$ and $B$.