Solved – Feature map for the Gaussian kernel

kernel trickmachine learningsvm

In SVM, the Gaussian kernel is defined as:
$$K(x,y)=\exp\left({-\frac{\|x-y\|_2^2}{2\sigma^2}}\right)=\phi(x)^T\phi(y)$$ where $x, y\in \mathbb{R^n}$.
I do not know the explicit equation of $\phi$. I want to know it.

I also want to know whether
$$\sum_ic_i\phi(x_i)=\phi \left(\sum_ic_ix_i \right)$$ where $c_i\in \mathbb R$. Now, I think it is not equal, because using a kernel handles the situation where the linear classier does not work. I know $\phi$ projects x to a infinite space. So if it still remains linear, no matter how many dimensions it is, svm still can not make a good classification.

Best Answer

You can obtain the explicit equation of $\phi$ for the Gaussian kernel via the Tailor series expansion of $e^x$. For notational simplicity, assume $x\in \mathbb{R}^1$:

$$\phi(x) = e^{-x^2/2\sigma^2} \Big[ 1, \sqrt{\frac{1}{1!\sigma^2}}x,\sqrt{\frac{1}{2!\sigma^4}}x^2,\sqrt{\frac{1}{3!\sigma^6}}x^3,\ldots\Big]^T$$

This is also discussed in more detail in these slides by Chih-Jen Lin of NTU (slide 11 specifically). Note that in the slides $\gamma=\frac{1}{2\sigma^2}$ is used as kernel parameter.

The equation in the OP only holds for the linear kernel.

Related Question