MATLAB: Optimization issue, always different results

MATLABoptimization

Hello everyone,
I'm trying to optimize a set of 3 parameters by using the following function:
options=optimset('Algorithm','levenberg-Marquardt', 'LargeScale','off', 'DiffMaxChange',0.01, 'DiffMinChange',0.0001, 'TolFun',1e-5, 'TolX',0.001);
[x,resnorm,residual,exitflag,output,lambda,jacobian]=lsqnonlin(@HSfun,x0,lb,ub,options);
At the end of the script, the function is reduced to a subtraction of vectors, so I do not consider important to post the whole script here. Input data is brought to the script from excel, and that's it. 3 parameters are optimized until reaching one of the optimization stopping criteria.
Altought I utilize Trust Reflective Region (default optimization Algorithm) instead of levenberg-Marquardt, the results of the optimization are always different for the same input data. And the results are sometimes more accurate than others.
My question is: Is this normal? These functions give you always different parameters answers even if the input data to optimize is the same?
Thanks in advance!

Best Answer

Yes, it is normal for different algorithms for lsqnonlin to give different results. lsqnonlin is not a global optimizer and the different algorithms have different search strategies. For any given starting point, there are functions for which a given algorithm may get stuck when the others do not.
Especially with non-linear functions, different algorithms may give different emphasis to different variables. That can result in seemingly very different values for a variable that turns out not to contribute much (and so the difference in function value is not high), and it can result in very different values for a variable that makes a significant difference in its best range, in the situation where there are locations where that variable does not make much difference and so might not have it range explored as much in one of the algorithms.
When you use lsqnonlin you should rarely assume that the result you get out is the global minimum.
Related Question