Consider the function $f:\mathbb{R}^d\mapsto\mathbb{R}$ given by
$$f(x) = x^{\intercal}a – \frac 12 x^\intercal B x~,$$
where $a\in\mathbb{R}^d$ and $B\in\mathbb{R}^{d\times d}$ is a symmetric positive definite matrix. We know that $f$ is minimized by the choice
$$x^* = B^{-1}a, \mbox{ and } f(x^*) = \frac{1}{2} a^\intercal B^{-1} a.$$
Now, consider the following analogous problem. Consider the function $f:L^2([0,1])\mapsto\mathbb{R}$ where $L^2([0,1])$ is the space of square integrable functions over $[0,1].$ For any $x:[0,1]\mapsto\mathbb{R},$ we have
$$f(x) = \int_0^1 x(u)a(u)\,du – \frac 12 \int_0^1 \int_0^1 x(u)B(u,v)x(v)\,dv\,du$$
where $a:[0,1]\mapsto\mathbb{R}$ is a square integrable function and $B(u,v) = B(|u-v|)$ is a symmetric positive definite kernel whose value depends only the absolute value of the difference between its two arguments. By Bochner's theorem, $B(\cdot)$ is the Fourier transform of a symmetric non-negative measure $\mu$ on the real line.
My questions:
a) What is the minimizer $x^*$ in closed form? Is there an analogous expression of the form $B^{-1}a$ in the linear algebra case?
b) You can assume that the measure $\mu$ is known and its density function exists and is available in closed form, if required. Can you write an analytical expression for $x^*$ using these?
c) Is there an analytical expression for the analog of $B^{-1}$, the "inverse" of a symmetric positive definite kernel?
b) If a closed form expression for $x^*$ is not available, is there a calculus-of-variations style condition that $x^*$ must satisfy?
Thanks a lot.
Best Answer
Here are some partial answers. The calculus of variations approach is to take a variation in the direction of another function $y(u)$, that is
$$\frac{d}{d\epsilon}\Big\vert_{\epsilon=0} f(x + \epsilon y) = 0.$$
After some work you get
$$\int_0^1 \left( a(u) - \int_0^1 B(u,v)x(v)\, dv\right) y(u) \, du = 0.$$
Since $y$ is arbitrary, the term in parentheses must vanish, and so the Euler-Lagrange equation for your problem is
$$\int_0^1 B(u,v)x(v)\, dv = a(u) \ \ \text{ for all } u\in [0,1].$$
This is the continuum equivalent of $Bx = a$. You can always define a mapping $B:L^2([0,1]) \to L^2([0,1])$ by
$$Bx[u]:=\int_0^1 B(u,v)x(v)\, dv,$$
and then your Euler-Lagrange equation is just $Bx =a$ and formally you can write $x = B^{-1}a$. Since you are assuming $B$ is symmetric and positive definite, the Fredholm theory should give you the existence of $B^{-1}$ just as an operator from $L^2$ to $L^2$. In general, I do not think it is possible to find an analytic expression for $B^{-1}$.