MATLAB: Can I set upper and lower bounds or constraints on the solution found by LSQNONLIN or LSQCURVEFIT

boundboundedconstrainconstraintconstraintsleastlsqcurvefitlsqnonlinOptimization Toolboxsquares

Can I set upper and lower bounds or constraints on the solution found by LSQNONLIN or LSQCURVEFIT? Can I solve a constrained least squares problem with the Optimization Toolbox?

Best Answer

As of MATLAB 5.3 and the Optimization Toolbox 2.0 (Release 11), the functions LSQCURVEFIT and LSQNONLIN allow you to set upper and lower bounds for your parameters.
If you want to apply constraints to your least-squares optimization, using FMINCON is a fairly reasonable approach to the problem. The methods most commonly used to solve nonlinear least squares problems, such as Gauss-Newton and Levenberg-Marquardt, assume the solution has a "small" or "zero" residual at the solution. It uses this assumption to efficiently solve the problem. If you try to use these methods on a large residual problem, then the method is slow (only linear convergence).
When you add constraints such as linear or nonlinear inequality/equality, there is no a priori way to know if you've constrained the problem to a region where the objective function always has a large residual - in other words, where the usual methods will be slow.
A faster alternative is to use a superlinear convergence method such as BFGS - which involves transforming the problem to a minimization problem just as you are doing.
There are some hybrid methods around to switch between, for example, Gauss-Newton and BFGS, but we aren't aware of any other methods that are substantially better than just using a constrained minimization routine as you are doing. This is why we recommend using FMINCON for a nonlinear least squares problem with constraints. You may want to run the optimizaiton several times with different initial conditions to find a better minima to improve the likelihood of finding the best minima.
Related Question