I'm not clear about kernel.
- How could I construct my own kernel that is valid? Is the only method the Mercer Theorem (positive semi-definite)? I mean if I know $K$ is a valid kernel, do I know that $aK+bK$, $\mathrm{polynomial}(K)$, $\exp(K)$, etc., are valid as well?
- If $K$ is a 2-dimension kernel, how could I expand (create) $K$ to 3-D?
- Given a kernel, say $K(x,y)=(x^Ty+1)^3$, how could I find a $\varphi$ that make this $K = \varphi^T \varphi$ hold?
Best Answer
Proving that the kernel is positive definite is sufficient. It's also true that various operations on kernels preserve kernel-ness; I recently catalogued some here, but many machine learning textbooks contain lists of such things.
One way is to use a product kernel: let $K_1$ be the kernel on the first two dimensions, $K_3$ one on the third dimension, and multiply them. There are also of course many other methods, depending on the specific kernel.
In general, this is quite difficult. In that specific case, you can expand out the polynomial and separate factors of various powers of each $x$ or $y$.