If the AUC score is 100 percent can the F1 value be 99.94 percent? I would expect 100 percent, too.
Machine Learning – Can AUC Score Be 100 Percent While F1 Value Is 99.94 Percent?
aucclassificationf1)machine learning
aucclassificationf1)machine learning
If the AUC score is 100 percent can the F1 value be 99.94 percent? I would expect 100 percent, too.
Best Answer
$AUC$ measures the separability of the probability outputs of your model. If the positive group's lowest probability of being positive is less than the negatives group's highest probability of being positive, then you will achieve $AUC=1$.
However, calculating $F_1$ requires you to apply a particular threshold (and only that threshold). Software usually picks that threshold to be a probability of $0.5$. The two groups need not be separable at $0.5$. It could be that every probability value (across both groups) exceeds $0.5$ or is lower than $0.5$.
Consequently, there should not be any expectation that threshold-based metrics be perfect when $AUC=1$.
If $𝐴𝑈𝐶=1$, then any threshold between the highest value for the negative group and the lowest value for the positive group results in classifying everything correctly, so all threshold-based metrics (at such a threshold) should be perfect (on the data set that generated the $𝐴𝑈𝐶=1$, not necessarily in general if you use additional data).