Solved – Naïve Bayes vs Bayes classifier

bayesianclassificationnaive bayes

While studying the naive Bayes classifier, two questions came to mind:

  1. When (if this can happen) is the naive Bayes classifier equivalent to the Bayes classifier?
  2. Why do I usually read in textbooks that naive Bayes is popular? What makes it so "popular"?

Best Answer

I would try not to conflate naive Bayes and the concept of a Bayes classifier. The former is a specific kind of classification model, whereas the latter should really just be viewed as an "optimal" classifier in a given setting (it could be any type of model, so long as it's the "true" model).

The reason we call the optimal classifier a Bayes classifier is because the best classifier needs to use Bayesian updating when making predictions, by which we mean that we follow Bayes theorem (it is a theorem after all) when updating our expectations based on evidence.

To say that a naive Bayes classifier is the Bayes classifier would just mean that no classifier can perform better in terms of misclassification rate (that is, it "knows" all the marginal distributions of the predictors for each class and correctly assumes they're all independent).

To your second question, I believe naive Bayes gained popularity because it's easy to implement and historically (despite it's generally false assumptions) it often performed well at certain tasks, particularly text classification.