Solved – Understanding Precision and Recall Results on a Binary Classifier

classificationmachine learningprecision-recallpython

I know the difference between Precision and Recall metrics in Machine Learning. One optimizes on False Positives and other on False Negative. In Statistics it is called as optimizing on Type I or Type II error.

However, I am a but confused on under what circumstances one can get complete opposite Precision and Recall? Like Precision =1 and Recall=0?.

Let me iterate: 

precision = true positives / (true positives + false positives)

recall = true positives / (true positives + false negatives)

And here is the Confusion Matrix

     predicted
            (+)   (-)
            ---------
       (+) | TP | FN |
actual      ---------
       (-) | FP | TN |
            ---------

Now, if Precision is =1 for a classifier for the positive (1) class, that means, there are no FP and all predicted labels are TP.

Then how can for the same positive class the Recall be 0? If there is already some TP being predicted, in fact as per Precision all predicted ones are TP only, then for Recall, we would have numerator non zero, then under what circumstances can one get Recall 0 then for the same classifier positive class?

To give some context, I ran a Logistic regression classifier for a binary classification problem. I had some 23K training data with 774 features. 770 features are binary or dummy variables.

And this is the distribution of my class labels: 

1    20429
0    12559

And here is the confusion matrix and accuracy values after a 5 fold grid search on some 25 combination of Hyper parameter values. 

The mean train scores are [ 0.66883049  0.54314532  0.67008959  0.63187226  0.63100366  0.53165968
  0.54131812  0.55507725  0.5578254   0.57663273  0.57247462  0.57230056
  0.54402055  0.5762753   0.50925733  0.45781882  0.39366017  0.39037968
  0.3919818   0.38878762  0.39784982  0.39506755  0.48238147  0.38932944
  0.39801223]

The mean validation scores are [ 0.66445801  0.54107661  0.66878871  0.63184791  0.6305487   0.5291239
  0.53899788  0.55324585  0.55822615  0.57784418  0.57269066  0.57312373
  0.54536399  0.57593868  0.50790351  0.45727773  0.39318349  0.38906933
  0.39214413  0.38924256  0.39794725  0.39461262  0.4827855   0.38811658
  0.39812048]

The score on held out data is: 0.6687887055562773
 Hyper-Parameters for Best Score : {'alpha': 0.0001, 'l1_ratio': 0.45}

The accuracy of sgd on test data is: 0.37526523188845107

Classification Metrics for sgd :
             precision    recall  f1-score   support

          0       0.38      1.00      0.55      3712
          1       1.00      0.00      0.00      6185

avg / total       0.77      0.38      0.21      9897

Best Answer

Then how can for the same positive class the Recall be 0? If there is already some TP being predicted, in fact as per Precision all predicted ones are TP only, then for Recall, we would have numerator non zero, then under what circumstances can one get Recall 0 then for the same classifier positive class?

Rounding. If you have one true positive, zero false positives, and the metrics themselves are rounded to two decimal places (as shown), precision is 1, recall is zero and accuracy is roughly 1/3 as shown.

Related Question