It's a known result that, under binormality assumptions, the area under the ROC curve (AUC) for a binary classifier has the following closed form. Formally, define the class conditional mean and variance for class $y \in \{0, 1\}$:
$$\mu_y = \mathbb{E}[X \mid Y = y]$$
$$\sigma^2_y = Var[X \mid Y = y]$$
Further assume $\mu_1 \geq \mu_0$. Then the AUC is given by
$$AUC = \Phi\left(\frac{\mu_1 – \mu_0}{\sqrt{\sigma_1^2 + \sigma_0^2}}\right)$$
where $\Phi$ is the Normal CDF. I recently encountered this form in Fong et. al. (2021). A search turns up variations on this form as well, for example Pan and Metz (1997) or Bandos, Guo and Gur (2018).
I'm having trouble coming up with the derivation. Here's what I've tried. First, I know false positive rate (FPR) and true positive rate (TPR), under the binormal assumption, are given at some threshold $\tau$ as
$$TPR(\tau) = 1 – \Phi\left(\frac{\tau – \mu_1}{\sigma_1}\right) = \Phi\left(\frac{\mu_1 – \tau}{\sigma_1}\right)$$
$$FPR(\tau) = 1 – \Phi\left(\frac{\tau – \mu_0}{\sigma_0}\right) = \Phi\left(\frac{\mu_0 – \tau}{\sigma_0}\right)$$
so that the AUC can be written as
$$\int_0^1 TPR(FPR^{-1}(x)) dx.$$
However, I'm getting stuck expanding that integral (or maybe this is not the right method to derive this). Substituting in the definition of TPR and FPR under our assumptions, I get something like
$$\int_0^1 TPR(FPR^{-1}(x)) dx= \int_0^1 \Phi\left(\frac{\mu_1 – FPR^{-1}(x)}{\sigma_1}\right) dx = \int_0^1 \Phi\left(\frac{\mu_1 – (\Phi^{-1}(1-x)\sigma_0 + \mu_0)}{\sigma_1}\right) dx.$$
At this point, I'm unsure how to proceed towards the expected closed form for the AUC. Any tips on how to proceed from here, or resources with the derivation (any form of the AUC will do)?
Best Answer
This comes directly from the derivation of the probabilistic interpretation of AUC. First, we know
$$AUC = \int_0^1 TPR(FPR^{-1}(x)) dx.$$
Through a change of variables $u := FPR^{-1}(x)$, we have
$$AUC = \int_{FPR^{-1}(0)}^{FPR^{-1}(1)} TPR(u) FPR'(u) du = \int_{\infty}^{-\infty} \Phi\left(\frac{\mu_1 - u}{\sigma_1}\right)f_0(u) du,$$
where $f_0$ is the density of the negative class (i.e. pdf of $\mathcal{N}(\mu_0, \sigma_0^2)$). We'll later use the notation $f_1$ for the density of the positive class. To be rigorous, implicitly, there's another substitution at this phase as $FPR'(u) = f_0\left(\frac{u - \mu_0}{\sigma_0}\right)$, but the term inside $f_0$ has derivative one, so the effect is an identity multiplication.
Then, we can write the CDF (noting the flipped bounds this time) as $$\int_{-\infty}^{\infty} \left(\int_{-\infty}^{\infty} \mathbf{1}[t < u] \cdot f_1(t) \;dt\right)\; f_0(u) du = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \mathbf{1}[t - u < 0] \cdot f_0(u)f_1(t) \;du \;dt$$
which is the CDF for a difference of normal distributions. This distribution has variance $\sigma_0^2 + \sigma_1^2$, and mean $\mu_1 - \mu_0$, which yields
$$AUC = \Phi\left( \frac{\mu_1 - \mu_0}{\sqrt{\sigma_0^2 + \sigma_1^2}}\right)$$
as needed.