MATLAB: Parameter Optimization using Simulated Annealing

optimizationparameter tunesimulated annealing

I am new to optimization and trying to understand the basics, so sorry if it is a dumb question. Is it possible to tune parameters (which is a search problem) of a classifier using simulated annealing or other optimization technique, just for an example optimum value of "k" in KNN (I know there is an automatic hyperparameter optimization for KNN)?

Best Answer

Sure, you can do anything you want. It might not be a good idea, but feel free.

Write an objective function that is, say, the cross-validation error rate for a particular parameter. If you have k as your parameter, and a cross-valudation partition c, then you might have

fun = @(k)kfoldLoss(fitcknn(X,y,'CVPartition',c,...
  'NumNeighbors',k));

The only problem with this is that k is an integer variable, and most optimizers (including simulannealbnd) work only with continuous parameters. But you could use mixed-integer ga to optimize this.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

Related Solutions

MATLAB: How to implement an equality and inequality constraint

You said both an equality AND an inequality constraint in the subject. But then you showed only the inequality.

The inequality is trivial.

A = ones(1,N);
B = E_Total;

MATLAB: Simulated Annealing – Parameters definition

If you are using simulannealbnd then there is no way to enforce integer constraints. ga is the only Global Optimization Toolbox that allows integer constraints.