[Math] If the Jacobian matrix is positive definite, does that imply that the optimization problem has a unique solution

matricesoptimization

My PhD adviser told me that
if the Jacobian matrix of the optimality conditions is positive definite,
then it implies that the optimization problem has a unique solution.
I was wondering what is the basis for this statement,
in other words, what theorem or proposition gives this statement.

Here is a simple example which illustrates the statement.

Let $a, b > 0$ be positive constants.
Our goal is to compute the $(p, q)$ which maximizes the objective
$$
ap – bp^2 + bpq – bq^2.
$$
The first-order optimality conditions are:
$$
a – 2bp + bq = 0, \\
bp – 2bq = 0.
$$
If we take the Jacobian matrix of the vector-valued function
which is the optimality conditions,
it gives us the matrix
$$
\begin{bmatrix}
-2b & b \\
b & -2b
\end{bmatrix}.
$$

If I were to change the signs of the optimality conditions,
I could get the Jacobian to be the positive definite matrix
$$
\begin{bmatrix}
2b & -b \\
-b & 2b
\end{bmatrix}.
$$

What mathematical theorem or proposition tells us that
because the Jacobian matrix is positive definite,
therefore the optimization problem has a unique solution?

Best Answer

Positive definiteness of the Hessian is equivalent to convexity for twice differentiable functions, and convexity is what's really at work here. So, the theorem you want is

Theorem Let $f:U\to\mathbb{R}$ be strictly convex, where $U\subset \mathbb{R}^n$ is convex. Then if $f$ has a local minimum in $U$, it has a global minimum.

You'll observe that looks weaker than your advisor's claim might have sounded: there are convex functions with no minimum, the simplest example being $e^x$. But it's easy to find local minima of differentiable functions, certainly in your case when the first derivatives form a linear system, so then you can be sure of a global minimum.

Related Question