I am working with a highly class-skewed three class classification problem. The class percentages are A = 1.8%, B = 17.5% and C = 80.7%. According to this paper, the following definition of multi-class AUC is insensitive to class distributions and therefore, I am using it:
Now I am testing 10 different classfiers on the dataset and only one of them has $AUC_{total} > 0.5$. Am I right in assuming that $AUC_{total} = 0.5$ for a random classifier? If yes, are 9 of the classifiers worse than a random classifier (on this data) just because they have a lower $AUC_{total}$? If yes again, can you suggest possible reasons for such poor performance and what can be done.
P.S.: Even the best performing classifier has $AUC_{total}$ of about 0.6.
Best Answer
I assume that to compute $AUC(c_i,c_j)$, you use the classifier obtained by comparing the probabilities of pertaining to these two classes.
Then, for the random classifier which draws uniformly a point $(p_1, \dots, p_n)$ in the $n$-simplex (I denote $n = |C|$), you have $AUC(c_i, c_j) = 0.5$.
As there are $n( n - 1)$ pairs $(c_i, c_j)$, if each $AUC(c_i, c_j) = 0.5$ then $AUC_{total} = 0.5$, yes.