MATLAB: Does Support Vector Regression (fitrsvm) optimization result in poor performance

optimizationStatistics and Machine Learning Toolboxsupport vector regression

Hi
I am working on a numerical prediction problem (load forecasting). I have a predictor matrix which consists of around 20 numerical variables (X_train: including historical lags, calendar information, temperature etc) and an output vector of real/target load values (Load_train). The data consists around 10.000 points
I am following the below documentation on Support Vector Regression, in particular the section 'Optimize SVM Regression'. https://matlabacademy.mathworks.com/R2016b/portal.html?course=mlml#chapter=4&lesson=7&section=3
However after the exhaustive 'hyperparameter optimization' I get poor prediction performance, especially compared to a simpler SVR which uses Gaussian Kernel, Kernel Scale 'auto' and all other parameters set to default without any optimization. I do standardize the input and output matrices before the training. Please find below the lines for two training procedure:
"Simple SVR"
mdl_simple=fitrsvm(X_train,Load_train,'Standardize',true,
    'KernelFunction','gaussian','KernelScale','auto');
"Optimized SVR"
Mdl_optimized = fitrsvm(X,Y,'Standardize','true','OptimizeHyperparameters','auto',...
    'HyperparameterOptimizationOptions',struct('AcquisitionFunctionName',
    'expected-improvement-plus'))
Would anyone have any advice with regards to the optimization procedure or any ideas why optimized SVR might be giving worse results than simpler SVR?
Best regards
Baran

Best Answer

Maybe it's because the optimized SVR uses the default kernel function, which is 'linear'. If you include 'KernelFunction','gaussian' in your second call to fitrsvm, as you did in your first, it might perform better.
Secondarily, you could try running the optimization longer by adding the field 'MaxObjectiveEvaluations' to your struct:
Mdl_optimized = fitrsvm(X,Y,'KernelFunction','gaussian','Standardize',true,'OptimizeHyperparameters','auto',... 'HyperparameterOptimizationOptions',struct('AcquisitionFunctionName', 'expected-improvement-plus', 'MaxObjectiveEvaluations', 60))
Related Question