I am evaluating a multi class classifier. As precision and recall are only defined for binary classification I want to create precision recall curves for every class by separating one class from all other classes. Using one-vs-all SVMs it’s very simple. For each class the appropriate SVM delivers a score which can be used with multiple Thresholds to create the precision recall curve. But how does it work with the nearest neighbor classifier? Is it possible to create a meaningful precision recall curve for the NN classifier? How can I use the distance values to create a score which can be combined with several thresholds?
Solved – Precision recall curve for nearest neighbor classifier
machine learningprecision-recall
Best Answer
The precision/recall curve for KNN classifier consists of two points effectively (since KNN predicts binary values) so such curve is not very useful or meaningful. One could instead use the fraction of a given class in the neighborhood (i.e. non-smoothed density estimate; requires K > 1) as the model prediction which would make the precision/recall curve more meaningful.
As for leveraging distance, one could try to use "distance-weighted voting" by relying on a distance-based kernel to get the weights (as one of the simplest kernel density methods) but that would not really qualify as KNN classifier since since KNN's kernel is just a step function.