MATLAB: How to apply majority voting for classification ensemble in Matlab

Question

I have five classifiers SVM, random forest, naive Bayes, decision tree, KNN,I attached my Matlab code. I want to combine the results of these five classifiers on a dataset by using majority voting method and I want to consider all these classifiers have the same weight. because the number of the tests is calculated 5 so the output of each classifier is 5 labels(class labels in this example is 1 or 2). I'll be gratefull to have your opinions

clear all
close all
clc
load data.mat;
data=data;
[n,m]=size(data);
rows=(1:n);
test_count=floor((1/6)*n);
sum_ens=0;sum_result=0;
test_rows=randsample(rows,test_count);
train_rows=setdiff(rows,test_rows);
test=data(test_rows,:);
train=data(train_rows,:);
xtest=test(:,1:m-1);
ytest=test(:,m);
xtrain=train(:,1:m-1);
ytrain=train(:,m);
%-----------svm------------------
svm=svm1(xtest,xtrain,ytrain);
%-------------random forest---------------
rforest=randomforest(xtest,xtrain,ytrain);
%-------------decision tree---------------
DT=DTree(xtest,xtrain,ytrain);
%---------------bayesian---------------------
NBModel = NaiveBayes.fit(xtrain,ytrain, 'Distribution', 'kernel');
Pred = NBModel.predict(xtest);
dt=Pred;
%--------------KNN----------------
knnModel=fitcknn(xtrain,ytrain,'NumNeighbors',4);
     pred=knnModel.predict(xtest);
     sk=pred;

how can I apply majority voting directly on these outputs of classifiers in Matlab?

Thanks very much

Answer 1

I don't think that there's an existing function that does that for you, so you have to build your own. Here is a suggested method:

• Assuming you have your five prediction arrays from your five different classifiers, and
• all prediction arrays have the same size = length(test_rows), and
• you have 2 classes: 1 & 2, you can do the following:

% First we concatenate all prediciton arrays into one big matrix. 
% Make sure that all prediction arrays are of the same type, I am assumming here that they
% are type double. I am also assuming that all prediction arrays are column vectors.
Prediction = [svm,rforest,DTree,dt,sk];
Final_decision = zeros(length(test_rows),1);
all_results = [1,2]; %possible outcomes
for row = 1:length(test_rows)
    election_array = zeros(1,2); 
    for col = 1:5 %your five different classifiers
       election_array(Prediction(row,col)) = ... 
           election_array(Prediction(row,col)) + 1;
    end 
    [~,I] = max(election_array);
    Final_decision(row) = all_results(I);
end

Hope this helps.

Ahmad