When reading this article, I noticed that the legend in Figure 3 gives a p-value for each AUC (Area Under the Curve) from the ROC (Receiver Operator Characteristic) curves. It says:
The area under the curve (AUC) is 1.0 (p < .001) for the overall
D-IRAP scores, 0.95 (p < .001) for the female picture bias scores and
0.94 (p < .001) for the male picture bias scores.
I'm helping a friend who is developing a similar study. To draw the ROC curves and calculate their AUC, I'm using the R pROC package, which has a function called roc() which returns a roc object. Analyzing this roc object, however, I couldn't find anything similar to a p-value. What am I missing? How did the authors of the cited article come to that p-value? I couldn't find anything in their methods. Does ROC curves usually come with a p-value?
Best Answer
No, it doesn't.
p-valuesfrom ROC areas are new to me, but I read the following:Mason and Graham (2002) show that the ROC area is equivalent to the Mann–Whitney U-statistic testing the significance of forecast event probabilities for cases where events actually occurred with those where events did not occur. Function
roc.area()of R-packageverificationapplies this approach and uses functionwilcox.test()to calculate thep-value. To me, it looks like this was done in the paper you referenced.