MATLAB: Nelder-Mead optimization with equality constraints

nelder-meadoptimization

Hello
I would like to use the function fminsearch of matlab to search for the best hyperparameters of my SVM with a weighted RBF kernel classifier. fminsearch uses the Nelder-Meader simplex method.
Let's say I have the following hyperparameters: C, gamma, w1….wn where wi are the weights of the kernel.
Additionally, I have the constraint that sum(wi) = 1, i.e. all weights must sum up to one.
Is there a possibility to use Nelder-Mead with this equality constraint? I know that there is the fminsearchbnd method for Matlab but I think it can handle only boundary inequality constraints.

Best Answer

I would imagine that you also have the constraints that several or all of your parameters must be positive.
It might be difficult to have all those constraints satisfied at the same time using just fminsearch, or even fminsearchbnd. You might want to use fmincon or patternsearch to ease your search for the best hyperparameters.
It is likely that you would do best by setting up a rough grid of hyperparameters, search them, and then start a local search using patternsearch or fmincon starting at the best grid point.
Good luck,
Alan Weiss
MATLAB mathematical toolbox documentation
Related Question