classificationclusteringmodel-evaluation
If the ground truth of the class/cluster/segment that our observations belong to, is known in advance, why would someone choose to perform clustering instead of classification? In fact, doesn't the problem "automatically" become a classification problem?
This question came to my mind as I was going through some clustering performance evaluation criteria. I came across the Rand index, an evaluation metric that requires the ground truth to be known in advance, hence my question.
Have a look at:
Vendramin, L., Campello, R. J., & Hruschka, E. R. (2010).
Relative clustering validity criteria: A comparative overview.
Statistical Analysis and Data Mining, 3(4), 209-235.
and follow-up work by the same authors. I believe I saw a table there that conained a "feature table" for the various indexes in literature, in particular whether they support overlapping clusters. But I could not find the table (and exact citation) right now.
I worked on this topic. Reference: Adjusting for Chance Clustering Comparison Measures
A one-line summary of the paper is: AMI is high when there are pure clusters in the clustering solution.
Let's have a look at an example. We have a reference clustering V consisting of 4 equal size clusters. Each cluster is of size 25. Then we have two clustering solutions:
AMI will choose U1 and ARI will choose U2.
Eventually:
If we are using external validity indices such as AMI and ARI, we are aiming at matching the reference clustering with our clustering solution. This is why the recommendation at the top: AMI when the reference clustering is unbalanced, and ARI when the reference clustering is balanced. We do this mainly due to the biases in both measures.
Also, when we have an unbalanced reference clustering with small clusters, we are even more interested in generating pure small clusters in the solution. We want to identify precisely the small clusters from the reference. Even a single mismatched data point can have a relatively higher impact.
Other than the recommendations above, we could use AMI when we are interested in having pure clusters in the solution.
Here I sketched an experiment where P generates solutions U which are balanced when P=1 and unbalanced when P=0. You can play with the notebook here.
Best Answer
You would want to cluster instead of classifying when the real-world problems don't share the same categories as the evaluation set you use.
For instance, let's say you know the true clusters of a small network into six groups. If you were to learn a classifier, then for all future networks you'd only be able to split them into six groups. By contrast, with clustering, you can divide them into arbitrary numbers of groups, which may be more appropriate.
You validate the clustering model on datasets that you know, in the hopes that it generalizes better to ones that you don't.
At a broader level, if the ground truth is known, then there's nothing left for you to predict—so the only reasonable goal is trying to understand (or explain) that structure. You can have competing hypotheses (competing models) of how that ground truth structure arose. Some might be drawn from clustering literature; others from classification literature. Each has its own set of inductive biases.