I think I should rephrase my question after reading a few replies. The original question is kept intact at the bottom. So maybe I should ask the question this way: if you use a ROC curve to find the predictive power of your predictor, will the slope on the ROC curve be a good metric to rely on?
Assume there is a undiscovered relationship f(x) = x(1-x), where 0 <= x <= 1. We also assume f(x) is a good predictor for our class label y. Now all you have is some (x, y) pairs. If you want to see how x predicts y, you generate a ROC curve and get your AUC, you would get a plot similar to what I have shown below, and my question about the predictive power of x on y remains the same.
Hope this is clearer.
——————- original question ——————————
Is it true that the slope of a ROC implies prediction performance, i.e., the bigger the slope a segment on the ROC curve is, the better the prediction the segment corresponds to.
Take a look at the following ROC curve
Can I assume the beginning part (TP rate 0~0.2) and the ending part (FP rate 0.85 ~ 1) predict much better than the middle segment?
The reason I am asking is that I am wondering if I can throw away data points in the middle and use only data points correspond to the segments at the two ends. Does this make sense?
Thank you!
p.s. I understand I could have reversed the predictor so that AUC > 0.5, but the questions remain the same.
Best Answer
Drawing the ROC curve leads to more confusion. Think about what is at the root of your issue. You can't interpret pieces of the ROC curve without having a utility/cost/loss function. The area under the ROC curve happens to equal the $c$-index (concordance probability) which is a simple interpretable pure measure of predictive discrimination.
Instead of saying something is a "good predictor of a class label" use "good predictor of the probability of class membership" and think hard about measuring the predictive discrimination in the model's predicted probabilities (especially if the model is well calibrated).
If you have correctly modeled the function form of $x$ (here by, for example, fitting a quadratic effect in a binary logistic model), and you get an AUROC ($c$-index) below 0.5, you have made a computational error.