Solved – Naive Bayes Classifier in R with class weights

carete1071naive bayesrtext mining

I'm searching for a Naive Bayes classifier in R where I can add a paramter for class weights. I need this, because my data is highly unbalanced. Eg.:

Class1: 1000 examples
Class2: 800 examples
Class3: 80 examples

I already tried the Support Vector Machine from the e1071 package which has such a paramter option and this worked quite well:

I already tried the Naive Bayes implementation in the e1071 as well as the caret package, but neither of them seem to have this possibility to define class weights.

Is there another implementation in R with this functionality or does anybody have suggestions how to achive this somehow different?

Thanks an BR

Best Answer

caret uses the naive Bayes function from the klaR package. It sounds like you want to adjust the prior:

prior: the prior probabilities of class membership. If unspecified, the class proportions for the training set are used. If present, the probabilities should be specified in the order of the factor levels.

If so, you can pass that to NaiveBayes via train

Related Solutions

Solved – R – Plotting a ROC curve for a Naive Bayes classifier using ROCR. Not sure if I’m plotting it correctly

The problem may lies here

pr <- prediction(pred, realResults)

You may transform "pred" and "realResults" to 0-1 vector by :

predvec <- ifelse(pred=="Lost", 1, 0)
realvec <- ifelse(realResults=="Lost", 1, 0)

and using:

pr <- prediction(predvec, realvec)

the problem may be solved.

Bonus part:

you can plot roc curve with more information by:

plot(ROCRperf, colorize=TRUE, print.cutoffs.at=seq(0,1,by=0.1), text.adj=c(-0.2,1.7))

And a simple way to get auc:

as.numeric(performance(ROCRpred, "auc")@y.values)

Solved – Naive Bayes error with caret

The problem lies in the fact that your data is highly imbalanced. If you look at the distribution of position, you will notice that FS and TE only appear once in your dataset. Since this is a factor the cross validation encounters no value for these 2 values, but expects them, because they are present in the factor level. Hence you will see in your warnings stop("Zero variances for at least one class in variables:", : missing value where TRUE/FALSE needed

If you remove these 2 values from your data you will see that the model will be created (with some warning messages).

data <- subset(data, !position %in% c("FS", "TE"))
data$position <- droplevels(data$position)

train_control <- trainControl(method="cv")
naiveModel <- train(position~groesse+koerpergewicht+bankdruecken+kniebeuge+maximalkraft_bb+maximalkraft_l+maximalkraft_r+schnellkraft_l+schnellkraft_r+sprint_10m+sprint_20m, data, trControl=train_control, method="nb")

To solve the imbalance you could use trainControl with sampling = "up" like this: train_control <- trainControl(method="cv", sampling = "up")

But better is to see if you can increase the number of records with position values TS Fand TE

Best Answer

Related Solutions

Solved – R – Plotting a ROC curve for a Naive Bayes classifier using ROCR. Not sure if I’m plotting it correctly

Solved – Naive Bayes error with caret

Related Question