I'm trying to use non-linear conjugate gradient method to minimize the following function
$$ f(x_1, x_2) = 100(x_2-x^2_1)^2 + (1-x_1)^2$$
Im not sure however how to figure out the step lengths $a_k$ in the non linear version of the method. Can I still consider what we call an exact line search and just take the $a$ that minimizes the objective function $f$ along the direction $x_k + a_k p_k$, where $p_k$ is my chosen direction?
If this is an option, then what is the point of inexact line search, in which you need to find step lengths that satisfy certain conditions, such as the Wolfe conditions?
Best Answer
You want that your line search only uses a very minimal number of steps, as typically in applications the function evaluations are expensive (solution of a PDE etc.). Thus the Wolfe and other conditions tell you when it is "good enough".
Note that this Rosenbrock function is a counter-example for the gradient descent method and all its variations, that is, it is expected that the method eventually stalls with very small step sizes far away from the true minimum. If I remember correctly, systematically overshooting the minimum of the line-search by some small percentage will visibly improve the convergence in this example.