Suppose $X$ is a compact metric space, and $k:X\times X\to\mathbb{R}$ is a continuous, universal kernel on $X$. By definition of a universal kernel, the RKHS corresponding to $k$, defined as
$$ \mathcal{H} = \overline{\mathrm{span}}\{k(x,\cdot):x\in X\}, $$
i.e., the closure of the span of the evaluation functionals with respect to the Hilbert norm $\|\cdot\|_{\mathcal{H}}$, is dense in $C(X,\mathbb{R})$ (the space of continuous functions on $X$) with respect to the sup-norm $\|\cdot\|_{\infty}$.
What I would like to know is how the span of the evaluation functionals relates to $C(X,\mathbb{R})$.
In particular, define the pre-Hilbert space
$$ \mathcal{H}_0 = \mathrm{span}\{k(x,\cdot):x\in X\}, $$
so that $\mathcal{H}=\overline{\mathcal{H}_0}$, where the closure is again taken with respect to the norm of the Hilbert space inner product.
The question that remains for me: is $\mathcal{H}_0$ dense in $C(X,\mathbb{R})$ with respect to the sup-norm $\|\cdot\|_\infty$?
I don't think that the Hilbert space norm is equivalent to the sup-norm, so I can't claim that the Cauchy sequences in $\mathcal{H}_0$ with respect to the Hilbert norm are also Cauchy with respect to the sup-norm, unless I am missing some property of universal kernels.
Best Answer
Since the kernel $k$ is assumed to be continuous and $X$ is assumed compact, there exists an $L>0$ such that for all $x\in X$, it holds that $k(x,x)>L$. Now, suppose that $(f_j)_{j=1}^\infty$ is a Cauchy sequence in $\mathcal{H_0}$ with respect to the norm $\|\cdot\|_{\mathcal{H}}$. Thus, for all $\epsilon>0$, there exists an integer $N$ such that for all $m,n\geq N$, we have
\begin{align*} \frac{\epsilon}{\sqrt{L}} &\geq \|f_m-f_n\|_{\mathcal{H}} \\ &= \sup_{g\in\mathcal{H}_0:\|g\|_{\mathcal{H}}\leq 1}\langle f_m-f_n, g\rangle \\ &\geq \sup_{x\in X}\big|\langle f_m-f_n, \frac{k(x,\cdot)}{\|k(x,\cdot)\|_{\mathcal{H}}}\rangle\big| \\ &= \sup_{x\in X}\big|\frac{1}{\sqrt{k(x,x)}}\big|\cdot\big|f_m(x)-f_n(x)\big| \\ &\geq \frac{1}{\sqrt{L}}\|f_m-f_n\|_\infty. \end{align*}
Therefore, $(f_j)_{j=1}^\infty$ is also Cauchy with respect to $\|\cdot\|_\infty$.
To complete the proof, let $g\in C(X,\mathbb{R})$ be given. Then, for any $\epsilon>0$, there exists an $f\in\mathcal{H}$ such that $\|g-f\|_\infty\leq\frac{\epsilon}{2}$. Since $\mathcal{H}$ is the completion of $\mathcal{H}_0$, there exists a Cauchy sequence $(f_j)_{j=1}^\infty$ in $\mathcal{H}_0$ that converges to $f$ in the norm $\|\cdot\|_{\mathcal{H}}$. By the argument above, it must also converge to $f$ in the sup-norm $\|\cdot\|_\infty$. Thus, there exists an $N$ such that for all $n\geq N$, it holds that $\|f_n-f\|_\infty\leq\frac{\epsilon}{2}$. By the triangle inequality, this implies that for all $n\geq N$, $\|f_n-g\|_\infty\leq\epsilon$, as desired.