MATLAB: How to compute kernels

kernelssvm

Hello

I want to calculate weighted kernels (for using in a SVM classifier) in Matlab but I'm currently compeletely confused.

I would like to implement the following weighted RBF and Sigmoid kernel:

x and y are vectors of size n, gamma and b are constants and w is a vector of size n with weights.

The problem now is that the fitcsvm method from Matlab need two matrices as input, i.e. K(X,Y). For example the not weighted RBF and sigmoid kernel can be computed as follows:

K_rbf = exp(-gamma .* pdist2(X,Y,'euclidean').^2)
K_sigmoid = tanh(gamma*X*Y' + b);

X and Y are matrices where the rows are the data points (vectors).

How can I compute the above weighted kernels efficiently in Matlab?

Can I just apply the weights before computing the kernel? And if yes, how can this be done?

Best Answer

instead of pdist2, that performs rest sqrt, and then back ^2, you could simply use the operators .^ and .* to operate element wise on W X and Y:

    K_rbf = exp(-gamma * sum(sum(W.*((X-Y).^2))))

you don't mention it, but gamma seems to be a scalar, so gamma does not need the element wise product .*

2 sum() because sum only sums along a given dimension.

Why do you write that K(x,y)=exp( .. is same as K(x,y)=tanh( .. ?

If you find this answer of any help solving your question,

please click on the thumbs-up vote link,

thanks in advance

John

jgb2012@sky.com

Related Solutions

MATLAB: How does SVMTRAIN determine Polynomial and RBF Kernel parameters (gamma,C) in the Bioinformatics Toolbox 3.5 (R2010a)

MATLAB does not calculate the kernel parameters by itself. If there are no given values by the user, MATLAB uses default values. For the polynomial kernel the default order is 3. But you can set any polynomial order you want by calling SVMTRAIN with additional parameter.

For example with this code you create a Polynomial Kernel of order 4.

svmstruct = svmtrain(data, groups, 'Kernel_Function', 'polynomial', 'Polyorder', 4)

The RBF Kernel behaviour is similar. MATLAB uses a Kernel of this form:

k_rbf(x,y) = exp(-1/(2*sigma^2) * ||x-y||^2).

In literature this Kernel is also referred as:

k(x,y) = exp(-gamma * ||x-y||^2)

If you didn't choose any value for sigma, SVMTRAIN uses the default value of sigma = 1. For example, you can set sigma to 0.2 by call SVMTRAIN in the following way:

svmstruct = svmtrain(data, groups, 'Kernel_Function', 'rbf', 'RBF_Sigma', 0.2)

The penalty parameter C is called BoxConstraintValue in MATLAB. SVMTRAIN uses a default value of 1, but you can set this value (e.g. to 0.8) by your own by using:

svmstruct = svmtrain(data, groups, 'Kernel_Function', 'rbf', 'RBF_Sigma', 0.2, 'BoxConstraint', 0.8)

If you want to find the optimal values for (sigma,C) you should use cross validation and grid search. The idea is the following:

1. generate disjunct subsets A_i from the main set A, such that the union of the A_i equals A (see CROSSVALIND in Matlab help)

2. in a loop try parameters for (sigma,C) (so called grid search)

3. generate for each subset A_i a support vector machine svm_i with the test parameter pair (sigma,C)

4. test each svm_i with the remaining subsets A_j, j not equal to i

5. take a look on the error rate and take the pair (sigma,C) with the lowest error rate for all pairs (sigma,C) you have tested (see SVMCLASSIFY and CLASSPERF in MATLAB help)

6. calculate the final svm with the so found pair (sigma,C) based on the whole set A

You have still some parameters to choose, e.g. the number of subsets A_i and the stepsize for C and sigma. These parameters depends on your given data set and problem, so it is up to you.

MATLAB: How to solve a non homogeneous diff equation numerically

k=23; 
m=9.1093821545*1e-31; 
w=sqrt(k/m);
hb=(6.6260695729*1e-34)/(2*pi);
E0=hb*w*.5;
M=@(x,y)[y(2);(hb*w*.5-.5*k*x^2)*y(1)];
x1=-sqrt(hb*w/k);
x2=-x1;
[m,n] = ode45(M,[x1 x2],[0.; 0.]);

Result will be that y(1) and y(2) are identically zero.

Best wishes

Torsten.

Best Answer

Related Solutions

MATLAB: How does SVMTRAIN determine Polynomial and RBF Kernel parameters (gamma,C) in the Bioinformatics Toolbox 3.5 (R2010a)

MATLAB: How to solve a non homogeneous diff equation numerically

Related Question