Calculating Norm in a Reproducing Kernel Hilbert Space (RKHS)

functional-analysishilbert-spacesreproducing-kernel-hilbert-spacesvector-spaces

I see a lot of texts describing properties of Reproducing Kernel Hilbert Spaces (RKHS), but I can't seem to find one that explains how to actually calculate the norm in the RKHS. From Wikipedia, it says:

A RKHS is a space of functions in which point evaluation is a
continuous linear functional. Roughly speaking, this means that if two
functions $f$ and $g$ in the RKHS are
close in norm, i.e., $\|f-g\|$ is
small, then $f$ and $g$ are also
pointwise close, i.e., $ |f(x)-g(x)|$
is small for all $x$. The reverse need not
be true.

Questions:

  1. It seems a RKHS is a space full of functions, so each element in it is a function? But I can treat these elements as points nonetheless?
  2. How does the concept of "norm" make sense in this context? A
    Euclidean norm is the distance between two points in space, but this is the distance between two functions in a space?
  3. Assuming I understand question 2, how can I go about actually calculating this norm? An example or formula would be most appreciated.

Best Answer

a RKHS is a space full of functions, so each element in it is a function?

Yes.

I can treat these elements as points nonetheless?

Yes. RKHS can be considered as a metric or topological space, and elements of such spaces usually are considered as points. This often provides helpful geometrical visualizations. But, moreover, RKHS is also a linear and normed space, and elements of such spaces usually are considered as vectors. This also provides sometimes even more helpful and concrete geometrical visualizations.

How does the concept of "norm" make sense in this context? A Euclidean norm is the distance between two points in space, but this is the distance between two functions in a space?

We are going opposite direction in this case. A Hilbert space $H$ has an inner product $\langle\cdot,\cdot \rangle_H$ which provides a norm $\|\cdot\|_H$ on it. For a real Hilbert space we have $\|f\|_H=\sqrt{\langle f,f \rangle_H}$ for each $f\in H$. So $H$ can be considered as a normed space endowed with a norm $\|\cdot\|_H$. It provides a distance $d_H$ on $H$ such that $d_H(x,y)=\|f-g\|_H$ for each $f,g\in H$.

how can I go about actually calculating this norm? An example or formula would be most appreciated.

This norm is provided with a Hilbert space.

A basic example is a generalization of a finitely-dimensional Euclidean space $\Bbb R^n$, namely a linear space $\ell_2$ of functions $f$ from $\Bbb N$ to $\Bbb R$ such that $\sum_{n\in\Bbb N} f(n)^2<\infty$. It is endowed with the inner product $\langle f, g \rangle=\sum_{n\in\Bbb N} f(n)g(n)$ for each $f,g\in\ell_2$. Then $\ell_2$ is a normed space with a norm $\|f\|=\sqrt{\sum_{n\in\Bbb N} f(n)^2}$ for each $f\in\ell_2 $ and a metric space with a metric $d(f,g)=\sqrt{\sum_{n\in\Bbb N} (f(n)-g(n))^2}$. Finally, $\ell_2$ is RKHS because for each $n\in\Bbb n$ an evaluation operator $L_n:\ell_2\to\Bbb R$, $f\mapsto f(n)$ equals to an inner product $\langle f, e_n\rangle$ of the function $f$ with the standard unit vector $e_n\in\ell_2$, such that $e_n(n)=1$ and $e_n(m)=0$ for each natural $m\ne n$.

Remark that in the previous paragraph the set $N$ can be replaced by any set $X$, providing a Hilbert space $\ell_2(X)$. When $X$ is finite then the spaces $\ell_2(X)$ are isomorphic to finitely-dimensional Euclidean spaces $\Bbb R^n$.

Related Question