Are the minima of both positive definite functions and positive semidefinite functions necessarily unique and therefore also global minima? Or is there a difference here that I'm missing?
I ask because I came across the following two questions, where the answers seem to suggest that this is the case for both types of functions:
Semi-positive definite Hessian matrix and local minimum
Since the hessian is positive semidefinite for all $x$, the function is convex (though not strictly convex). So the stationary point is a minimum, and a global minimum in fact (by convexity). Think of it this way – the function is increasing in the direction of the eigenvector with eigenvalue 5, and flat in the direction of the eigenvector with eigenvalue 0. If the Hessian were negative semidefinite, you would have a global maximum.
Quadratic Function must be positive definite to have a unique minimum
We know that:
A twice differentiable function of several variables is strictly convex on a convex set if and only if its Hessian matrix is positive definite on the interior of the convex set.
Any local minimum of a convex function is also a global minimum
A strictly convex function will have at most one global minimum.
So, basically, to guarantee that $V$ has a unique minimum we need its Hessian to be positive definite.
We have that $x = \left( {{x_1}, \ldots ,{x_n}} \right) \in {\mathbb{R}^n}$, so $V = V\left( x \right) = V\left( {{x_1}, \ldots ,{x_n}} \right).$
$$V\left( x \right) = a + {b^T}x + \frac{1}{2}{x^T}Cx = a + \sum\limits_{i = 1}^n {{b_i}{x_i}} + \frac{1}{2}\sum\limits_{i = 1}^n {\sum\limits_{j = 1}^n {{c_{ij}}{x_i}{x_j}} } $$
$$\frac{\partial }{{\partial {x_k}}}V\left( x \right) = {b_k} + \frac{1}{2} \cdot 2\sum\limits_{i = 1}^n {{c_{ki}}{x_i}} $$
$$\frac{\partial }{{\partial {x_l}}}\left( {\frac{\partial }{{\partial {x_k}}}V\left( x \right)} \right) = {c_{kl}}$$
Thus, the Hessian of $V$, which by definition has entries $${\left( {{H_V}\left( x \right)} \right)_{i,j}} = \frac{\partial }{{\partial {x_j}}}\left( {\frac{\partial }{{\partial {x_i}}}V\left( x \right)} \right)$$
is
$${H_V}\left( x \right) = C.$$Hence, for $V$ to have a unique global minimum, $C$ has to be positive definite.
I would greatly appreciate it if people could please take the time to clarify this.
Best Answer
For a convex function, a local minima must be a global minima.
However, this doesn't means that the local minima is unique.
For example, $f(x)=0$ is a convex function and any $x$ is a global minima. The $f$ that I stated is just positive semidefinite. Positive definiteness gives us unique solution but positive semidefinite function need not.