I am trying to understand the AUC (Area under the ROC curve) in the context of evaluating the performance of an algorithm for doing item recommendation(e.g. BPRMF). I know how the calculation is performed, but I am not able to show it graphically. For example, for calculating AUC for single user u, the following procedure is performed:
- Given positive and negative items for certain user u, we calculate the performance of the model by using the prediction value of the positive item, and the prediction values of all negative items. Then, for each negative item, we check whether the prediction value of the positive item is higher than the prediction value of the negative item. If it is higher, then we count it as a correct prediction (considering that positive items are more preferred than the negative items). Then, the total number of correct predictions is divided by the total number of negative items. Then we would repeat the same procedure for the next positive item, and sum up at the end the values in each iteration. At the end, we would have the AUC value for certain user.
Can anyone help and show me how one could present this graphically?
You can find the formulation of the AUC with regard to ranking context in the following paper: Bayesian Personalized Ranking.
Best Answer
The AUC computation shown in the paper is a short cut. You can actually plot ROC curve with usual TPR & FPR as axis. The points will vary based on recommendation set size 'k'. For an explanation of same, you can refer to - https://wiki.epfl.ch/edicpublic/documents/Candidacy%20exam/Evaluation.pdf