By Sara Garzone and Nicola Ruschena
In the context of text mining, topic modelling analyses co-occurrence patterns among textual data, in order to isolate clusters from the set of expressions occurring in a corpus. Topic modelling aims at extracting topics occurring in a corpus and categorize documents on the basis of their semantic content. It often represents an appealing approach for data-driven analysis in short-run projects, for it is an unsupervised method, i.e., there is no requirement for algorithm training from labelled data, whose production is quite a demanding task. Moreover, software programs that are executable from command line or user interface have been developed to perform topic modelling, so as to provide more friendly environments for researchers who are not much acquainted with code design.
Mallet is a tool for topic modelling: it is a Java-based package for statistical natural language processing, which was initially developed by Andrew McCallum at the University of Massachusetts. It allows topic modelling on textual corpora, without requiring advanced technical knowledge in statistics and programming.
Mallet’s topic modelling is based on the Latent Dirichlet Allocation (LDA) model, a Bayesian probabilistic generative model which has been applied for the first time to text classification tasks by David Blei et al. in 2003, and thereafter has become the standard for probabilistic text categorization under latent semantic hypotheses. Along with many other techniques in the field of natural language processing, topic modelling relies upon the so-called distributional hypothesis (Harris 1954), according to which words occurring in the same contexts tend to have similar meanings.
From co-occurrence analysis and clustering it is then possible to expect clusters to reflect semantic proximity relations, or topics. With advanced applications of probabilistic models, a categorization of documents can then be obtained on the basis of the degree of probability of their being a member of detected topics. The underlying assumption is that in each document a probabilistic distribution of every topic can be recognized. With LDA-based topic modelling one can try to understand which of the topics that have been detected in the corpus are likely to be present in each document, given the occurring terms.
Continue reading →