I try to classify physiological signals into 3 classes by using the fitcecoc routine in Matlab R2016b. I have configured the function in order to automatically optimize the hyperparameters of the trained SVM. It seems to work pretty fine. However, I have experienced some difficulties in understanding how this magical tool works.
It is claimed in Matlab help that the optimization is performed through a k-fold cross-validation process. I think I understand this part: 3 SVMs (in my case) are trained k times on (k-1)/k of the dataset and tested on the remaining 1/k each one providing a binary loss, in fine, the best hyperparameters are chosen in order to minimize a loss-weighted function.
So, could we say that fitcecoc returns an optimized ECOC SVM object that has been cross-validated? (afterall, it performs a k-fold cross-validation within its optimization algorithm)
It is not clear for me since one can also specify cross-validation input arguments to fitcecoc (that are specifically used for cross-validation) and which have the same effect as both crossval function and method.
Is there any interest to perofm a classical cross-validation after fitcecoc?
Many thanks in advance for enlightening me!
Best Answer