MATLAB: How to find the Multi-Class Hyperplane Decision Boundaries using Support Vector Machines (SVM)

boundariesdecisionhyperplaneStatistics and Machine Learning Toolboxsvm

How can I find the multi-class decision boundaries of a hyperplane using Support Vector Machines?

Best Answer

Example on Multi-Class Hyperplane Decision Boundaries
Please refer to the following example, illustrating the required steps for 3 different classes. The example can also be found in the attached "multiclassHyperplaneDecisionBoundary" MATLAB Live Script.
Create and visualize random data with 3 classes
M  = 200;
X = [mvnrnd([-1,0],2*eye(2),M); ...
    mvnrnd([2,2],eye(2),M); ...
    mvnrnd([-1,4],eye(2),M)];
y = [ones(M,1); 2*ones(M,1); 3*ones(M,1)];
figure;
plot(X(y==1,1),X(y==1,2),'.','DisplayName','1'); hold on
plot(X(y==2,1),X(y==2,2),'.','DisplayName','2');
plot(X(y==3,1),X(y==3,2),'.','DisplayName','3'); hold off
Create a multiclass error-correcting output codes (ECOC) model for support vector machines (SVM) Create an SVM template, and standardize the predictors
template = templateSVM( 'Standardize', 1);
Except for the StandardizeData, Method, and Type properties, most of the template object's properties are empty. When training the ECOC classifier, the software sets the applicable properties to their default values.
Use the specified predictor data in X and the response data in y to train the ECOC model:
ecoc = fitcecoc(X,y,'Learners',template);
Use separating hyperplane equation to classify sample data A compact SVM object for binary classification can be obtained from the trained ECOC model by indexing the 'BinaryLearners' array:
idx = 1;
svm = ecoc.BinaryLearners{idx};
The hyperplane separating two classes, for an observed sample x = (x1,x2), can be found by setting its function f(x) = 0. The separating hyperplane equation is given by
f =@(x) ((x(1)-svm.Mu(1))/svm.Sigma(1))*svm.Beta(1)/svm.KernelParameters.Scale + ...
    ((x(2)-svm.Mu(2))/svm.Sigma(2))*svm.Beta(2)/svm.KernelParameters.Scale + svm.Bias;
Note that svm.Beta/svm.KernelParameters.Scale can be simplified to svm.Beta since
svm.KernelParameters.Scale
ans =
     1
The relative location of a test sample w.r.t. the separating hyperplane can found by evaluating the separating hyperplane equation and looking at the sign of the obtained value:
testSample = mean(X) + std(X).*randn(1,2);
testSVMClass = sign(f(testSample))
testSVMClass =
     1
The relation between binary SVM class names and the corresponding ECOC input data class can be easily found with the following command:
[svm.ClassNames ecoc.ClassNames(ecoc.CodingMatrix(:,idx)~=0)]
ans =
    -1     1
     1     2
Please note that the the predict function (<https://mathworks.com/help/releases/R2018a/stats/compactclassificationsvm.predict.html documentation link>) can also be used for this purpose:
testSVMClassPred = predict(svm,testSample)
testSVMClassPred =
     1
Plot decision boundary for each binary classification SVM in the ECOC model Following this approach, the decision boundaries for all ECOC model input classes can be calculated following the calculation procedure below:
for i = 1:length(ecoc.BinaryLearners)
    svm = ecoc.BinaryLearners{i};
    % Step 1: Choose hyperplane's x1 values
    x1vals = linspace(min(X(:,1)),max(X(:,1)));
      % Step 2: Solve for ((x2-svm.Mu(2))/svm.Sigma(2)) in terms of x1.
      x2fun =@(x1) -svm.Beta(1)/svm.Beta(2) * ((x1-svm.Mu(1))./svm.Sigma(1)) - svm.Bias/svm.Beta(2);
      % Step 3: Calculate corresponding x2 values
      x2vals = x2fun(x1vals) * svm.Sigma(2) + svm.Mu(2);
      hold on;
      plot(x1vals,x2vals,'--', ...
          'DisplayName',mat2str(ecoc.ClassNames(ecoc.CodingMatrix(:,i)~=0)'));
      hold off;
  end
  legend('show','Location','eastoutside','Orientation','vertical');
Related Question