MATLAB: Decrease step size in lsqnonlin() for presentation.

diffmaxchangelsqnonlinoptimization

I am using the lsqnonlin optimization function to minimize an objective function with ~50 variables. I'm also making a movie (using OutputFcn with getFrame) to show what is happening the these variables at each iteration (for presentation purposes).

Currently, the algorithm does ~20 iterations before it stops so my video ends up with 20 frames. 20 frames isn't enough if I want a nice smooth movie that is at least 5 seconds long. I figure I need roughly 120 frames (5 sec. * 24 FPS) to get a nice smooth video. The only way I can think to do this is to force the optimization algorithm to do at least 120 iterations.

What options could I play with to force more iterations from lsqnonlin? DiffMaxChange maybe? Note that I don't want to just have it continue doing iterations after a solution is found. Rather, I want the objective function to change less between iterations.

Any help or insight would be greatly appreciated. I don't know much about how these optimization algorithms work.

Regards, Justin

Best Answer

So you want a less efficient solver, one that takes more function evaluations to find an optimum? I think you are making a mistake in your thinking. You should be happy that lsqnonlin takes so few steps.

You have many options for making a movie. You could show 6 frames of each current frame. Or you could interpolate results, if that makes sense for what you are trying to do, perhaps coloring things different for the interpolated frames.

If you really want to mess up lsqnonlin, and have it converge slowly, or not at all, give an erroneous Jacobian and set the 'Jacobian' option to 'on'. But I think this will not show how the solver works when it is working well.

Alan Weiss

MATLAB mathematical toolbox documentation

Related Solutions

MATLAB: Fmincon Current Step Size

The only solution that worked for me was to provide user-defined Hessian function. Although it took me few weeks to calculate and code second derivatives, since the software will be used many times by many people - it was worthwhile. With correct Hessian function fmincon works like charm.

Btw. I believe that there is a room for improving fmincon algorithm when it comes to "step size". It was obvious in my case when it stalled at certain value. Although with each subsequent iteration the value of the objective function was constantly decreasing (without oscillations), the rate was way too slow. I suggest adding some logic that will increase step size when objective function constantly decreasing without oscillations.

Thanks

MATLAB: 1 iteration in fsolve: new x and no more

One option is to use fsolve just to get the Jacobian and then generate x1 yourself. This gives you access to JacobPattern and all the other Jacobian calculation features that fsolve offers.

To do so, replace your objective function f(x) with f(x)-f(x0), where x0 is thte initial point. Note that this does not change the Jacobian. Because x0 is a root of the modified objective, fsolve will return x0 and its Jacobian as output in zero iterations. Example:

fun=@(x) x.^2/2;
options=optimoptions(@fsolve,'MaxIter',1);
x0=rand(5,1);
f0=fun(x0);
[x,fval,ef,stats,JacobianNumerical]=fsolve(   @(x)fun(x) - f0 ,x0,options);
Equation solved at initial point.

fsolve completed because the vector of function values at the initial point
is near zero as measured by the value of the function tolerance, and
the problem appears regular as measured by the gradient.
JacobianNumerical,
JacobianNumerical = 5×5
    0.3167         0         0         0         0
         0    0.5539         0         0         0
         0         0    0.7692         0         0
         0         0         0    0.0906         0
         0         0         0         0    0.5526
JacobianAnalytical=diag(x0)
JacobianAnalytical = 5×5
    0.3167         0         0         0         0
         0    0.5539         0         0         0
         0         0    0.7692         0         0
         0         0         0    0.0906         0
         0         0         0         0    0.5526

Also, it will do this with no extra Jacobian calculations, as seen from,

stats
stats = struct with fields:
       iterations: 0
        funcCount: 6
        algorithm: 'trust-region-dogleg'
    firstorderopt: 0
          message: '↵Equation solved at initial point.↵↵fsolve completed because the vector of function values at the initial point↵is near zero as measured by the value of the function tolerance, and↵the problem appears regular as measured by the gradient.↵↵<stopping criteria details>↵↵Equation solved. The final point is the initial point.↵The sum of squared function values, r = 0.000000e+00, is less than sqrt(options.FunctionTolerance) = 1.000000e-03.↵The relative norm of the gradient of r, 0.000000e+00, is less than options.OptimalityTolerance = 1.000000e-06.↵↵'

Related Question

How to change patternsearch options for paretosearch algorithm