I understand that AUC under ROC curve is a classic evaluation measurement for classifiers (which is basically the accuracy). However, when data is imbalanced, PR will be alternative. So, what does the AUC under PR curve mean?
Also, for example, if I obtain an optimal threshold for classifier in ROC curve (like the one that minimize the error), can I use that optimal threshold and calculate the Precision and Recall in PR-curve?
Best Answer
Area under the ROC curve is equivalent to concordance (aka $c$-statistic) (not accuracy!). This can be interpreted as the probability that a random positive is assigned a higher score than a random negative. Unfortunately, area under the PR curve has no such interpretation (that I'm aware of).
The relationship between ROC and PR curves stems from the fact that both are based on the same source: contingency tables for every possible decision value threshold. Every threshold $T$ leads to a contingency table (e.g. $TP^{(T)}$, $FP^{(T)}$, $TN^{(T)}$, $FN^{(T)}$).
Every point in ROC space is based on a certain decision threshold $T$, and therefore coincides with a point in PR space. If a given model's ROC/PR curve dominates another, that model's PR/ROC curve will also dominate (cfr. Davis & Goadrich).
Two remarks: if you want to select the threshold which minimizes the error (maximizes accuracy), ROC curves are not necessary (in fact they don't even show that). Secondly, if you have decided on a threshold you can just use the corresponding contingency table to get whatever other measures you want directly. Don't bother computing a full PR curve to then select 1 point of it.
Keep in mind that neither ROC or PR curves show you which threshold yields a certain point in the given space. They just show you the possible tradeoffs the model is capable of. That said, you can obviously map a point in ROC/PR space to a threshold if you retain a record of what thresholds they correspond to (most software packages to do this anyway).