Let $A$ be a symmetric matrix which is positive-definite. Let $\langle d_1,d_2 \rangle = d_1^TAd_2$ and $||d|| = \sqrt{\langle d, d \rangle}$. How can I determine the normalized direction of the steepest descent of a function $f$? In case of the euclidean Norm the solution is $d=-\frac{\nabla f(x)}{||\nabla f(x) ||}$, but I'm stuck finding out a solution for the given norm.
Steepest descent with other norm
analysisnonlinear optimizationoptimization
Best Answer
The direction of steepest descent is the unit vector that minimizes the directional derivative, i.e. $d$ should minimize $\langle\nabla f, d\rangle = \nabla f^T A d$. This direction is when $Ad = -\nabla f$. As $A$ is positive definite we can solve this system to get $d = -A^{-1}\nabla f$. As said before $d$ is only a direction, so we should normalize it to be unit length.
$||d|| = \sqrt{\langle -A^{-1}\nabla f, -A^{-1}\nabla f \rangle} = \sqrt{\nabla f^TA^{-1}A A^{-1}\nabla f} = \sqrt{\nabla f^TA^{-1}\nabla f}$.
Then the direction of steepest descent is $\frac{d}{||d||}$.