Machine Learning – Calculating Average of Precision and Recall

Question

A machine learning model is outputting precision and recall for a two-class classification problem (0 and 1) like this:

Confusion matrix:
[[136  21]
  41   6]]

Precision: [0.768 0.128]
Recall: [0.866 0.222]
Accuracy: 0.696

There are two measures for both precision and recall: The first measure for the 0 class and the second for 1 class. Is it okay to take the average of these? E.g. precision as a whole is 0.768+0.128 / 2 = 0.448? And similarly with recall?

Answer 1

WARNING: Average of precision/recall is totally different concept from Average Precision(AP) link.

Based on the question, we will talk about the Average of precision and recall.

you are partially correct; if

      Predicted 0 , Predicted 1 
True 0 [[136             21]     [[TP  FN]
True 1   41              6]]       FP  TN]]

Then precision for each class(row) is ( Mi,i / sigma(j) Mji), So for:

class 0: 136/ 136+41 (0.76)

class 1: 6 / 6+21 ( 0.22)

for recall, the same happens, but the denominator will be on rows, i.e. ( Mi,i / sigma(j) Mij)

class 0: 136/ 136+21 (0.86)

class 1: 6 / 6+41 ( 0.12)

Then you can average on each group to have overall precision/recall.

Check Table III of this paper (referred to as Precision_M and recall_M):

More precisely, you are doing macro-averaging.

in code, you can have :

cm = confusion_matrix(labels, predictions)
recall = np.diag(cm) / np.sum(cm, axis = 1)
precision = np.diag(cm) / np.sum(cm, axis = 0)

#overall precision/recall
np.mean(precision)
np.mean(recall)

Readmore