Let $f: \mathbb{R}^N \rightarrow \mathbb{R}$. The Newton direction $s=-[\nabla^2f(x)]^{-1}\nabla f(x)$ a descent direction if:
$\nabla f(x)^Ts<0 \implies -\nabla f(x)^T[\nabla^2f(x)]^{-1}\nabla f(x)<0 $
$\quad\quad\quad\quad\quad\quad\implies \nabla f(x)^T[\nabla^2f(x)]^{-1}\nabla f(x)>0$
$\quad\quad\quad\quad\quad\quad\implies [\nabla^2f(x)]^{-1} $ must be positive definite.
We know that the inverse of a matrix positive definite is also positive definite.
My question is: $\nabla^2f(x)>0$ is a necessary or sufficient condition? and why?
Thanks in advance for any help!
Best Answer
$\nabla^2 f(x) \succ 0$ is equivalent to "$v^\top [\nabla^2 f(x)] v > 0$ for any nonzero vector $v$."
However, you are only asking that the latter condition hold for a specific vector, namely $v = \nabla f(x)$. So positive-definiteness of the Hessian is sufficient for the step to be a descent direction, but not necessary.
(Recall that you run this step many times in Newton's method for different values of $x$ that you do not know in advance. One usually assumes the Hessian is positive-definite everywhere so that we have this assumption that the step is a descent direction. Otherwise it is quite hard to certify whether or not a step will be a descent, and more complex analyses of $f$ would be required to assess how Newton's method will behave.)