[Math] Why Should One Use Backtracking Line Search Method to Implement an Algorithm

algorithmsMATLABnewton raphsonoptimization

I am new to MATLAB and I am asked to implement on matlab the following algorithm:

Steepest descent
Newtont
Quasi-Newton (bfgs)
Gauss-Newton

using a line search method and the justify my decision.
I am using the backtracking line search cause I know that to use that I just need to saisfy one of the Wolfe conditions and to be honest also because it's the only method we studied so far, so I don't know how to justify properly my choice.
Could you please give me some more explenations?
What could I say more?

Best Answer

Using backtracking, assuming the direction selected is a descent direction, guarantees improvement in the minimization in each iteration.

In many cases it is sufficient, given enough steps, to get to local stationary point.

When not using Backtracking but, for example constant step size, one might diverge. Namely the step size should be set according to the surface being minimized and backtracking is one of the simplest way to do so.

Related Question