In machine learning we may use the area under the ROC curve (often abbreviated AUC, or AUROC) to summarise how well a system can discriminate between two categories. In signal detection theory often the $d'$ (sensitivity index) is used for a similar purpose. The two are closely connected, and I believe they are equivalent to each other if certain assumptions are satisfied.
The $d'$ calculation is usually presented based on assuming normal distributions for the signal distributions (see wikipedia link above, for example). The ROC curve calculation does not make this assumption: it is applicable to any classifier that outputs a continuous-valued decision criterion that can be thresholded.
Wikipedia says that $d'$ is equivalent to $2 \text{AUC} – 1$. This seems correct if the assumptions of both are satisfied; but if the assumptions are not the same it's not a universal truth.
Is it fair to characterize the difference in assumptions as "AUC makes fewer assumptions about the underlying distributions"? Or is $d'$ actually just as widely applicable as AUC, but it's just common practice that people using $d'$ tend to use the calculation that assumes normal distributions? Are there any other differences in the underlying assumptions that I've missed?
Best Answer
No. The maximum value of the AUC is 1. d' has no maximum.
I believe that d' is equal to the qnorm(AUC)*sqrt(2) (my memory of an old stats book I can't find right now but seems to check out against some data I found on the web). Here qnorm(x) is the "quantile function for the normal distribution" (R-speak). That is, it returns the value of the normal distribution for which x proportion of the distribution is below it.