Solved – Standardisation in Naive Bayes

bayesiandata preprocessingmachine learningnaive bayes

Is it possible that the accuracy of Naive Bayes remain the same even after applying Standardisation . I have applied 2 Standardisation techniques :

  1. Min Max Scaling ( which squishes the range from 0-1 )

  2. Standard Scaling ( which makes the mean=0 and standard deviation =1 )

My implementation is correct as I checked for the range , mean and standard deviation after Standardising . However , the accuracies I obtain are exactly the same . Is this possible or is there some kind of error with my implementation ?

Best Answer

I assume you are using Gaussian, not Multinomial/Binomial Naive Bayes?

For Gaussian Naive Bayes, the estimator learns the mean and standard deviation of each feature (per class). At prediction time the probability of a value being in a class is a function of the distance from the center of the distribution. The function used is Probability Density Function (PDF), of a Normal/Gaussian distribution. And the Normal PDF is just a Standard Normal distribution (0 mean, unit variance) that is scaled by variance and shifted by mean. So a value which is at mean+(0.5*std) has the same probability.

With standardization the mean and stddev changes, but probabilities stay exactly the same, and thus classification results. In essence Gaussian Naive Bayes performs standardization internally.