It is well known that the area under the curve ($AUC$) is equal to the Mann-Whitney U statistic (c.f. Why is ROC AUC equivalent to the probability that two randomly-selected samples are correctly ranked?). Therefore it has to be true that $AUC$ is asymptotically normal. However, we have that $AUC$ is bounded from below by zero and bounded by one from above – a normally distributed random variable has unbound support though. How is this possible?
AUC – Asymptotic Distribution of AUC
asymptoticsaucnormal distributionwilcoxon-mann-whitney-test
Related Solutions
Ok, I found the answer and as I expected it is trivial. $U$ test statistic value depends on the group it is calculated for (it does not affect the test result in anyway). In the code I wrote the test statistic was computed as a measure of support for the hypothesis that the group with the smaller mean dominates the group with the higher mean, which is of course not true, so that's why $U$ was small.
So after switching the direction of the comparison and making the hypothesis tested by the Wilcoxon-Mann-Whitney test to one checking whether the group with the higher mean dominates the one with the lower, which is true, I got the correct relationship between $U$ and $AUC$ (that is $AUC = \frac{U}{n_1n_2}$). So everything is correct.
AUC can be viewed as Wilcoxon-Mann-Whitney Test. And here is some demo, where for the R code I posted, I first calculate AUC, then use Wilcoxon-Mann-Whitney Test to calculate the number. Then verify both numbers are the same which is 0.911332. For a hypothesis testing, it is not hard to derive confidence interval. Right? Also I do not remember it Wilcoxon-Mann-Whitney Test requires normal distribution.
Best Answer
Mann-Whitney's $U$ is not equal to ROC AUC, it is proportional to the area under the ROC curve:
$$\text{AUROC } = \frac{U}{n_1n_0}$$ where $n_0$ is the number of negative examples and $n_1$ is the number of positive examples and $U$ is the Mann-Whitney $U$ statistic.
From this expression, it should be clear that even $U$ must be bounded, because $\text{AUROC}\in [0,1]$, therefore $U \in [0, n_1 n_0]$. Therefore, the same "boundedness" problem that you describe in your question exists for the $U$ statistic.
I think what you mean is that $U$ is approximately normal (for some definition of "approximate") in the case of a large sample size. The Wikipedia entry has a nice description of how a $U$ statistic that is approximately distributed as a normal distribution due to a large sample size can be standardized. https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test#Normal_approximation_and_tie_correction
It's important to keep in mind the advantages and disadvantages of using an approximation. It's not hard to find that there are obvious disagreements between the normal approximation and the true distribution $U$. If your view is that the resulting error from approximation is too large for a particular purpose, then it's perfectly reasonable to not use the approximation and instead proceed with an approximation with lower error, or an exact method. On the other hand, these alternatives can be expensive, or even intractable, to compute, or simply more complex than we have time to implement and validate. Chiefly, approximations are a bargain, trading precision for convenience.