Solved – How to choose an internal clustering evaluation metric

clustering

I am trying to cluster textual data using fastText vectors with different clustering algorithms, mainly K-Means and DBSCAN.

I would like to know which internal evaluation metric works best with K-means and DBSCAN (ex: silhouette coefficient).

More specifically, I am looking for a metric that does not give higher values to convex-shaped like the silhouette coefficient in order to be able to compare clustering obtained with K-Means and DBSCAN.

My understanding is that internal clustering metrics are not only to be used for comparing a different number of clusters or clustering methods. What decision should I make if different metrics contradict each other i.e if one metric indicates an improvement and another, a decrease in clustering quality?

Best Answer

Actually almost no internal metric can handle DBSCAN results properly.

The problem is that noise is not a cluster and almost all metrics assume the data is strictly partitioned into disjoint clusters.

Most implementations will then evaluate noise like a cluster, and the result will appear much worse than it is.

Pretty much the only metric I know (but haven't used) for this is DBCV, it supposedly is designed for density based cluster evaluation.

In general the only way to choose an evaluation metric is to understand what it does. Pick there meric whose formal approach is most closely related to your desire of a "good" cluster. Because everybody seems to have a slightly different understanding of when a cluster is "good".