MATLAB: Computational Efficiency for Decoupled Optimisation

fminconfsolveMATLABoptimizationOptimization Toolbox

Hi all

Just looking for some best practice advice for setting up an optimisation study with a large number of variables. The cost function consists of multiple sub-models that don't interact with each other, but the outputs of all of them are summed to provide the cost value. Does anyone know if it's more computationally efficient to have one large optimisation process or to optimise each of the sub-models separately?

Thanks for your help,

Martin

Best Answer

You haven't mentioned any constraints, so I'm assuming fminunc would apply to your problem. If you do have constraints, note that they too would have to be independent for each sub-model in order for the overall optimization to be decouplable.

Assuming you have the memory to store the gradient and Hessian in sparse form, it should be more efficient to solve simultaneously, however, you would have to set up the option parameter input judiciously.

https://www.mathworks.com/help/optim/ug/fminunc.html#butpb7p-options

It would be necessary to use the trust-region reflective algorithm, which allows you to take advantage of the block sparsity of your Hessian, via either the HessPattern, HessianMultiplyFcn, or HessianFcn options. Note that with the trust-region method, you are also required to supply your own gradient calculation (SpecifyObjectiveGradient=true).

Related Solutions

MATLAB: Which algorithm converges faster

It is hard to know in advance which algorithm will be faster. Generally, the more information you can give to an algorithm (explicit gradient, Hessian sparsity, etc.), the faster the computation, but there are counterexamples. The fmincon trust-region-reflective algorithm is essentially the same as the fminunc trust-region algorithm, so there is no point trying both. But the fmincon sqp and interior-point algorithms are different, though sqp is not large-scale.

One of the most important things is to give a good initial guess, and that is often very difficult to obtain.

If the problem has no constraints, but you can prove that the solution lies in a bounded region, then it is almost certainly better to give the bounds explicitly and to use fmincon.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

MATLAB: Wrong Hessian output in fminunc

Another solution would be to use standalone routines for numerical gradient and hessian estimation

https://www.mathworks.com/matlabcentral/fileexchange/13490-adaptive-robust-numerical-differentiation

fun=@(x) (x(1)^2+x(2)^2);
xOpt = fminunc(fun ,[1,1]);
g=gradest(fun,xOpt)
H=hessian(fun,xOpt)

No hidden scale factors if you go that route:

xOpt =
     0     0
g =
     0     0
H =
    2.0000         0
         0    2.0000

Best Answer

Related Solutions

MATLAB: Which algorithm converges faster

MATLAB: Wrong Hessian output in fminunc

Related Question