Find a descent direction at a saddle point

Question

question:
(Descent directions at stationary points). Assume that $f: \mathbb{R}^n \rightarrow \mathbb{R}^n$ is $C^2$ with $\nabla f(x) = 0$ and $\nabla^2f(x)$ indefinite (with both positive and negative eigenvalue(s)) for some point $x\in \mathbb{R}^n$. Show that $f$ admits descent directions (in the weak sense) at $x$.

Answer:
if $\nabla^2f(x)$ is indefinite, then $x$ is a saddle point. This means that there exists a direction of $f$ at the point $x$ and $d\in \mathbb{R}^n$ such that there exists $\epsilon >0$ with

$$
\forall \lambda \in (0,\epsilon) f(x + \lambda d) < f(x)
$$

What is missing to prove that $f$ admits a descent direction at $x$?

Answer 1

As you pointed out, when the Hessian $\Delta^2 f(x)$ is indefinite there exists some vectors $ d \in \mathbb R^n$ sucht that

$ d^T\Delta^2 f(x)d>0 $

and others such that

$d^T\Delta^2 f(x)d<0$.

Moreover, as you considered $\Delta f(x)=0$, we are treating saddle points. So you can see that we are in front of two admissible directions :

• Descent direction i.e. $\qquad f(x+\lambda d)<f(x)$$\qquad\forall \lambda\in (0,\epsilon), \,x,\,d\in \mathbb R^n $
• Uphill direction i.e. $\qquad f(x+\lambda d)>f(x)$$\qquad\forall \lambda\in (0,\epsilon),\,x,\,d\in \mathbb R^n $

You can easily visualize it on this link. The black dot $\bullet$ is the saddle point where your gradient equals zero, then on $A B$ you obviously have uphill directions starting at $\bullet$, whereas and on $C \bullet D $ you have descent directions. Therefore, you can conclude without any further assumptions that $f$ admits descent directions.