In linear algebra, the kernel of a matrix is its null space.
In machine learning and statistics, there are a bunch of matrices are called "kernel". For example,
I am totally confused. The second "kernal" concept looks very much like a projection to me, rather than a "null space". Why they are given the same name? Is there any connection between the two concepts?
Thank you!
Best Answer
I would say that there is just a lot of over loading of the word kernel. In some areas it means something similar to the linear algebra definition e.g.:
Kernel (linear algebra), the set of all vectors which map to the zero vector Kernel (set theory), the set of all pairs of elements that map to the same value
But in other areas such as integral transforms and machine learning it is more used in the sense of the 'nucleus' or 'key ingredient' to a certain transform or operation. For example in machine learning a given co-variance function model is called a kernel.
My advice would be to not try and link all of these things. There are many different objects in maths called the same thing for no reason other than they were developed separately and no-one was checking to see what already might have that name. Often it also comes down to word origins, apparently in German there is the term "der Kern" which translates to "the core", also a kernel is a single grain on a plant. To me it seems that the first old definition could give rise to the machine learning type usages of the word and (since mapping to zero is like reducing lots of things to a single grain) the second definition could give rise to the linear algebra type usage. But like I said I wouldn't worry too much exactly why they are used in these ways.
P.s. some namings in maths are because there are strong links between the usages which are important to understand, so it is always good to check as you have done!