My PhD adviser told me that
if the Jacobian matrix of the optimality conditions is positive definite,
then it implies that the optimization problem has a unique solution.
I was wondering what is the basis for this statement,
in other words, what theorem or proposition gives this statement.
Here is a simple example which illustrates the statement.
Let $a, b > 0$ be positive constants.
Our goal is to compute the $(p, q)$ which maximizes the objective
$$
ap – bp^2 + bpq – bq^2.
$$
The first-order optimality conditions are:
$$
a – 2bp + bq = 0, \\
bp – 2bq = 0.
$$
If we take the Jacobian matrix of the vector-valued function
which is the optimality conditions,
it gives us the matrix
$$
\begin{bmatrix}
-2b & b \\
b & -2b
\end{bmatrix}.
$$
If I were to change the signs of the optimality conditions,
I could get the Jacobian to be the positive definite matrix
$$
\begin{bmatrix}
2b & -b \\
-b & 2b
\end{bmatrix}.
$$
What mathematical theorem or proposition tells us that
because the Jacobian matrix is positive definite,
therefore the optimization problem has a unique solution?
Best Answer
Positive definiteness of the Hessian is equivalent to convexity for twice differentiable functions, and convexity is what's really at work here. So, the theorem you want is
You'll observe that looks weaker than your advisor's claim might have sounded: there are convex functions with no minimum, the simplest example being $e^x$. But it's easy to find local minima of differentiable functions, certainly in your case when the first derivatives form a linear system, so then you can be sure of a global minimum.