Solved – R caret difference between ROC curve and accuracy for classification

Question

In case of caret package test function metric option, one can use either accuracy or ROC as a metric that will be used to finalize values of tuning parameters. I felt that accuracy and ROC are the same

Accuracy = total correct predictions/total samples

ROC= looks at various probability cutoffs and gives probability cutoff at which accuracy will be the best.

Am I correct?

Answer 1

Generally speaking, Area Under the RoC (AUROC) statistic is used when you have imbalanced classes. For ex: 5% 1's and 95% 0's.

In practice, we are more interested in the AUROC to judge how well the model rank orders cases (i.e., rank from high probablity to low probablity of being a 1) where as Accuracy is... well you already know that.

In the context of model tuning, my advice would be to use AUC (especially if you have imabalanced classes) instead of Accuracy.