I'm trying to prove that if the Hessian $A$ of $f:\mathbb{R}^n\rightarrow\mathbb{R}$ is positive definite at $a$, then the function attains a minimum at $a$.
I can't figure out how to use the definition itself, $x^TAx>0$. So I'm trying to use the fact that since the Hessian is symmetric, that being positive definite is equivalent to all the eigenvalues being positive, and also using the fact that a symmetric matrix has eigenvectors which form an orthonormal basis for $\mathbb{R}^n$.
So basically I'm trying to show that for any vector $x\in\mathbb{R}^n$, $Ax$ does not change the sign of any of the components of the vector. I'm pretty sure this is a valid way to show that curvature is positive in all directions, but I'm not positive.
Anyways since every $x\in\mathbb{R}^n$ can be written as in terms of the orthonormal basis of eigenvectors of $A$. I have $Ax = \lambda_1(x\bullet b_1)x + … + \lambda_n(x\bullet b_n)x$.
Can anyone straighten out my understanding of this problem?
Edit: This is for $n\geq 2$ and the Hessian invertible.
Best Answer
It is not true. The function $f(x) = \frac{1}{2} \|x\|^2$ has Hessian $I$ which is positive definite everywhere, but only $x=0$ is a minimum.
If you assume that $a$ is a stationary point of $f$, and that $f$ is twice continuously differentiable at $a$, then the statement is locally true.
Taylor's theorem gives $f(a+h) = f(a) + \frac{ \partial f(a) }{ \partial x } h + \frac{1}{2} \langle h, \frac{ \partial^2 f(\xi) }{ \partial x^2 } h \rangle$, for some $\xi$ on the line between $a$ and $a+h$.
We have assumed that $a$ is a stationary point, hence $\frac{ \partial f(a) }{ \partial x } = 0$. Since $\frac{ \partial^2 f(a) }{ \partial x^2 } > 0$, we have $\frac{ \partial^2 f(\xi) }{ \partial x^2 } > 0$ for $\xi$ in some neighborhood $U$ of $a$. Hence for $\xi \in U$, we have $\langle h, \frac{ \partial^2 f(\xi) }{ \partial x^2 } h \rangle \ge 0$. Combining these gives $f(a+h) \ge f(a)$ for all $a+h \in U$. Hence $f(a)$ is a local minimum.
In fact, the above can be tweaked slightly to show that, in fact, $a$ is a strict local minimizer:
Let $\phi(x,h) = \langle h, \frac{ \partial^2 f(x) }{ \partial x^2 } h \rangle$, and $\eta(x) = \min_{\|h\| = 1} \phi(x,h)$. A little work shows that $\eta$ is continuous at $x=a$, and $\eta(a) >0$. Hence for some neighborhood $V$ of $a$, and some $\lambda>0$, we have $\eta(x) \ge \lambda > 0$ for all $x \in V$. It follows from this that $\phi(x,h) \ge \lambda \|h\|^2$ for $x \in V$.
Then, for $a+h \in U \cap V$, we have $f(a+h) \ge f(a) + \lambda \|h\|^2$.