Solved – How to plot results from text mining (e.g. classification or clustering)

classificationclusteringdata visualizationpythontext mining

In text classification and clustering, the number of features are normally big, e.g. I currently get are around 5,000 features which is already really small compared to many other text mining tasks. Given I am totally new to visualization, I have no clue about how should I plot the results from text classification and clustering.

For example, in one clustering task, I have three categories, hundreds of documents, and some thousands of features in these texts. After I apply a cluster, e.g. the KMeans cluster from scikit-learn, I have no idea about how to plot these results onto a 2D figure. It is the same for classification, say, I get texts sorted into three categories, but I get no idea how to plot them.

I've tried to learn from some examples, but most ones I found are based on pure data with very few features rather than text.

Question: Could you refer me to any tutorial or paper on either drawing text clustering/classification results or explaining the math part of such visualization. Or any help would be appreciated.

Best Answer

If you're doing classification, this should be fairly straight forward. Just select some aggregate measure of performance (e.g. accuracy), and plot a distribution of that measure for different random initializations of k-means. This gives you some information about how well the algorithm would perform on average.

If you're doing clustering (i.e. unsupervised clustering), then you can make a pretty picture of the clusters using a vector compression technique. A simple technique might be to pick a point in space, and plot each point in your dataset with its euclidean distance from that point and its category. You could also use more advanced techniques like PCA.

If you're interested in finding out which features are good predictors, I suggest running something like a maximum entropy classifier on it, or one of many feature selection algorithms. These techniques will provide you with a weight for each feature indicating its importance in predicting the groupings.

Related Solutions

Solved – Assigning meaningful cluster name automatically

One technique for this is unsupervised multi-document keyword extraction. That is, extracting the most salient words and/or collocations for each of your clusters. A number of methods are available to do that, most of which are centered around either using graph centrality measures (think: Google's PageRank) or using language processing (NER/noun phrase detection) of each document, or simply using all possible n-grams and then extracting the most significant entities/phrases according to, e.g., their log(TF)-IDF scoring or log(TF)-Entropy score. Instead of using all n-grams, you could also detect collocations (ideally, with a likelihood ratios estimator, p. 161 ff.) on a large, independent corpus, and then use those (only) to form any higher n-gram.

However, in your case, maybe, statistical (feature) selection techniques might be more appropriate, like Boostrap testing to extract the terms (again, either sampling over all possible n-grams or by selecting specific, pre-computed collocations) that are significant for each cluster, relatively to all others. Note that you can use either the raw word counts or plug in any re-weighted count from the TF-IDF family of term weighting functions mentioned above. However, these statistical techniques require that you have access to all words (viz, word counts) among all clusters and indeed have the computational resources to do an all against all comparison, something that might be infeasible if your clusters and word data are too large.

Solved – Text Mining: how to cluster texts (e.g. news articles) with artificial intelligence

Latent Dirichlet Allocation (LDA) is great, but if you want something better that uses neural networks I would strongly suggest doc2vec (https://radimrehurek.com/gensim/models/doc2vec.html).

What it does? It works similarly to Google's word2vec but instead of a single word feature vector you get a feature vector for a paragraph. The method is based on a skip-gram model and neural networks and is considered one of the best methods to extract a feature vector for documents.

Now given that you have this vector you can run k-means clustering (or any other preferable algorithm) and cluster the results.

Finally, to extract the feature vectors you can do it as easy as that:

from gensim.models import Doc2Vec
from gensim.models.doc2vec import LabeledSentence

class LabeledLineSentence(object):
    def __init__(self, filename):
        self.filename = filename
    def __iter__(self):
        for uid, line in enumerate(open(self.filename)):
            yield LabeledSentence(words=line.split(), labels=['TXT_%s' % uid])


sentences = LabeledLineSentence('your_text.txt')

model = Doc2Vec(alpha=0.025, min_alpha=0.025, size=50, window=5, min_count=5,
                dm=1, workers=8, sample=1e-5)

model.build_vocab(sentences)

for epoch in range(500):
    try:
        print 'epoch %d' % (epoch)
        model.train(sentences)
        model.alpha *= 0.99
        model.min_alpha = model.alpha
    except (KeyboardInterrupt, SystemExit):
        break

Best Answer

Related Solutions

Solved – Assigning meaningful cluster name automatically

Solved – Text Mining: how to cluster texts (e.g. news articles) with artificial intelligence

Related Question