MATLAB: Find optimal hyperparameters in SVM

hyperparametersvmtuning hyperplane

Hello
I'm trying to optimize a SVM model for my training data then predict the labels of new data with it. Also I must find SVM with best hyperparameter by using k-fold crossvalidation. TO do so I wrote the following code:
Mdl = fitcsvm(trainingData,labels,'OptimizeHyperparameters','auto',…
'HyperparameterOptimizationOptions',struct('Optimizer','gridsearch','AcquisitionFunctionName',…
'expected-improvement-per-second','MaxObjectiveEvaluation',10,'ShowPlots',false,'Verbose',0));
label = predict(Mdl,testData);
the problem is every time I ran this code and calculated the classification accuracy for test data I got different classification accuracy.
I should mention that when I train SVM without optimizing hyperparameters results are alaways the same. Is this mean every time I have diffierent hyperparameters? How can I solve this and obtain unique classification accuracy?

Best Answer

Read what the bayesopt documentation has to say about your chosen acquisition function: "Acquisition functions whose names include per-second do not yield reproducible results because the optimization depends on the runtime of the objective function. " In other words, choose the 'expected-improvement-plus' or 'expected-improvement' acquisition function for reproducibility.
Alan Weiss
MATLAB mathematical toolbox documentation
Related Question