I have a Hessian matrix $\mathbf{H}$ of a function $f(\mathbf{x})$, evaluated at an extreme point $\mathbf{x}_0$. Lets assume $\mathbf{H}$ is non-singular.
I can show that there exists a direction $\mathbf{z} \neq \mathbf{0}$ for which $\mathbf{z}^T \mathbf{H} \mathbf{z}$ is zero. Does this imply that the Hessian is indefinite and $f$ has a saddle point at $\mathbf{x}_0$?
My geometric intuition is that a saddle has cross-over lines between positive and negative regions. A minimum or maximum would not have such cross-over lines.
Moreover, I would argue that since the Hessian is assumed to be non-singular, $\mathbf{z}$ is not in its kernel, so $\mathbf{H} \mathbf{z} \neq \mathbf{0}$, so the expression $\mathbf{z}^T \mathbf{H} \mathbf{z}$ is not generally zero, but only in some directions.
Best Answer
Yes, your hunch is correct. The point is that as it was given that $\bf H$ is non-singular, the semi-definite cases are ruled out. Therefore the existence of a non-zero vector $\bf z$ such that $\bf{ z Hz^T}=0$ means that $\bf H$ must have eigenvalues of both signs, and we have a saddle-point.
More details. The matrix $\bf H$ is symmetric, hence diagonalizable. So with respect to an eigenbasis the quadratic form $$Q({\bf z})={\bf zHz^R}=\sum_{i=1}^n \lambda_i z_i^2.$$ Non-singularity means that $0\neq\det {\bf H}=\prod_{i=1}^n\lambda_i$. Therefore $\lambda_i\neq0$ for all $i$. So for $Q$ to vanish non-trivially it is necessary that there are cancellations among the terms $\lambda_i z_i^2$. This means that there must exist indices $i_1, i_2$ such that $\lambda_{i_1}>0$ and $\lambda_{i_2}<0$, and we are in the indefinite case.