I am beginning to tackle geostatistics problems where I tried to apply kriging(gaussian processes) to interpolate demographical water drop. According to my understanding, kernel methods are something like this: I define a feature mapping from original space to feature space. kernel function is the function of the two vectors from original space to the dot product of their according mapped vectors in the feature space. Once we have defined the kernel functions, it is very easy to define distance and angel in the feature space. In terms of this, kernel functions can be used to estimate similarity.Thus I see kernel tricks in machine learning as a method to reduce high-dimensional problems to low dimensions.But I kind of do not get a intuition of how kernel is actually applied to these machine learning problems, like gaussian processes, svm.Look forward to getting some intuition for this method.
I am also a bit confused about the weighted linear regression, gaussian processes and gaussian linear regression after reading a tutorial.(http://www.cs.nyu.edu/~mohri/icml2011-tutorial/tutorial-icml2011-1.pdf) I see gaussian processes as a locally method because of the bandwidth. But I do not know why it is also gaussian? We do not make assumptions on the prior distributions of the parameters.I just see gaussian processes as a constrained optimization problem.
Best Answer
I would recommend you to review the following resources on the topic with excellent explanations:
Nice blog post on kernel functions for machine learning (ML);
Another nice blog post on using kernel approximations / kernel trick in ML;
Yet another nice blog post on using kernel methods in ML;
Great discussion on Cross Validated (CV) on advantages of Gaussian kernel in ML.
For more detailed technical treatment of the topic, see references in the above-mentioned blog posts as well as in answers within the CV discussion, linked above. Hope this helps.