MATLAB: SVM with Dumthe Variables

dummy variablesmachine learningStatistics and Machine Learning Toolboxsvm

Context: I have a cell array with 19 features that are all categorical (nominal) (as columns) and ~1500 data entries (as rows). I've looped through all the columns and used double(dummyvar(nominal(featureVector))) to convert all the features into dummy variables (vectors of 1s & 0s) and all looks right.

Problem: When I try to feed this as the input data X to fitcsvm() it gives me an error as it expects X to be a floating point matrix.

    Error using ClassificationSVM.prepareData (line 602)
    You can pass only floating-point data for X to SVM.

If I convert the cell array into a matrix, then the dummy variable vectors will be represented as columns and thus they lose their identity as dummy variables as fitcsvm() expects each column to be a predictor in itself and now thinks there are (num of features)*(num of categories in each feature) predictors. So I don't see how I can use dummy variables with an SVM in Matlab which is mind boggling and I know this is a basic problem many will have.

Thanks so much for your help!

Best Answer

Just convert your cell array into a matrix. Yes, dummy variables will lose their identity in the sense that different levels of a categorical predictor will be treated as different predictors. This is common practice though.

Related Solutions

MATLAB: Sequentialfs with “dummified” input feature matrix

To answer this particular question:

"Is there any way instead of selecting category choices like this to select initial features? For example, I have features Weight, Model_Year, Color, Owner_Name, and Owner_Country and I want to know which of these are most predictive, not which of their category choices are."

I don't know why you would want to do this because selecting categories is in principle more useful than selecting predictors. But if you insist on doing that, just dummify variables inside your my_fun_lib. This works as long as all your features are categorical. Here is an example:

%%Form a table with two categorical variables
load fisheriris
T = table(categorical(meas(:,1)),categorical(meas(:,2)));
%%Convert table into matrix
X = zeros(size(T));
for p=1:size(X,2)
    X(:,p) = double(T{:,p});
end
%%Turn into a binary classification problem
y = strcmp(species,'setosa');
%%Run sequentialfs
c = cvpartition(y,'k',5);
opts = statset('display','iter');
inmodel = sequentialfs(@mycrit,X,y,'cv',c,'options',opts);

where mycrit looks like this:

function val = mycrit(Xtrain,Ytrain,Xtest,Ytest)
Ntrain = size(Xtrain,1);
X = dummyvar([Xtrain; Xtest]);
Xtrain = X(1:Ntrain,:);
Xtest  = X(Ntrain+1:end,:);
obj = fitcsvm(Xtrain,Ytrain);
val = loss(obj,Xtest,Ytest);
end

MATLAB: Reference dumthe coding with Matlab fitlme

Hi Ryan,

You can use categorical or nominal to specify the first category. Here's an example:

    % 0. Dummy data.
    rng(0,'twister');
    y = rand(30,1);
    g = [zeros(10,1);ones(10,1);2*ones(10,1)];
    T = table(y);
    % 1. First category is automatically set based on sort order. In this
    % case it will be 0.
    T.g = categorical(g);
    lme = fitlme(T,'y ~ g')       
    % 2. Make 2 the first category.
    T.g = categorical(g,[2,0,1]);
    lme = fitlme(T,'y ~ g')
    % 3. Same as 1 but using nominal.
    T.g = nominal(g);
    lme = fitlme(T,'y ~ g')
    % 4. Same as 2 but using nominal.
    gn = nominal(g);
    getlabels(gn)
    gn = reorderlevels(gn,{'2','0','1'});
    T.g = gn;
    lme = fitlme(T,'y ~ g')

Hope this helps,

Gautam

Related Question