Classical Topic Models
摘要
Topic modeling aims to discover topics in text data, where each topic can be represented by a distribution over the vocabulary, with weights denoting the relevancy of words to the topic. By selecting a small number of words with the largest weights associated with every topic (aka top words), we can obtain word clusters that reflect the latent semantics of topics. For instance, a topic on computer hardware could be captured by a cluster of words such as “scsi,” “bios,” “drives,” “controller,” and “bus.” In classical topic models, one stream of research focuses on mining diverse topics. The corresponding methods assume that topics are independent of each other. The other researchers concentrate on revealing the hierarchical relationships among topics, which are often observed in web pages and academic repositories. This chapter will summarize the classical topic modeling methods from the following two categories: flat topic models and hierarchical topic models. For the former, we introduce the flat topic models that utilize Siamese neural networks, neural mixed counting models, parallel Gamma-Gamma-Poisson processes, and variational auto-encoders, respectively. For the latter, we describe the hierarchical topic models that organize topics into various structures, including a tree, a forest, and more flexible graphs.