In the arcticle Fixed-Time Stable Gradient Flows: Applications to Continuous-Time Optimization I found an interesting formula and its properties. The screenshot of the page from the article I was led below.
In another article A Continuous Version of Newton's Method, I found a similar formula, but in contrast to the first article, where the ratio of the gradient $G$ to the hessian $H$ is used, i.e. $G/H$, this use $f(x)/G(x)$. Screenshot from the article I also led below.
Problem: I decided to "play" with formula
$\frac{dx}{dt}=-(\frac{d^2f}{d^2x})^{-1}\frac{df}{dx}$
and function $f(x)$
$f(x)=e^{-(x-x_*)^2}$
and found that none of them work as it should (you need convergence from the starting point $x(0)$ to the point $x_*$).
Below I give code from Mathematica and what happened.
Clear["Derivative"]
ClearAll["Global`*"]
pars = {xstart = -1, xend = 1}
f = Exp[-(x[t] - xend)^2]
E^-(-1 + x[t])^2
sys =
NDSolve[{x'[t] == -(D[D[f, x[t]], x[t]])^-1 D[f, x[t]],
x[0] == xstart}, {x}, {t, 0, 500}]
Plot[{Evaluate[x[t] /. sys], xend}, {t, 0, 25},
PlotRange -> Full, PlotPoints -> 100]
Question: What's wrong with this formula or where did I make a mistake?
I will be glad any help.
Best Answer
In the screenshoted page, theorem 3 ensures convergence to the optimal point $x_*$ in fixed time if assumptions 1 and 3 are satisfied.
Assumption 1 (not on the screenshot) ask for the optimal value $x_*$ ($f(x_*)$ is the minimum of $f$) to be reached at finite values. However, since you took $f(x)=e^{-(x-x_*)^2}$, $f(x_*)=1$ is actually the maximum and the minimum of the function, $0$ is reached at $\pm \infty$ (explaining why your solution seems to decrease to $-\infty$).
Considering $f(x)=-e^{-(x-x_*)^2}$ instead should do the trick if the initial point is close enough to $x_*$ for the assumptions of the theorem to be verified (this function is strictly convex only near it and this is required by assumption 3).