Is it possible for the Gauss-Newton algorithm to converge to a local minima? How does convergence of Gauss-Newton to the correct minima compare with gradient decent and Levenberg-Marquardt?
Gauss-Newton local convergence
nonlinear optimizationnumerical methods
Best Answer
All three of those are "local" methods, which means that they will be attracted to local minima if they start close enough to them even if they are not global minima.
Non-convex optimization is just difficult in general. There is no general recipe to follow to guarantee getting the correct minimum in a reasonable amount of computation time. (Certain stochastic methods will always work eventually, but keeping runtime down is difficult.)