Affiliations: [a] National Key Laboratory for Novel Software Technology, Department of Computer Science and Technology, Nanjing University, Nanjing, Jiangsu, China | [b] School of Digital Media, Jiangnan University, Wuxi, Jiangsu, China
Correspondence:
[*]
Corresponding author: Chong-Jun Wang, National Key Laboratory for Novel Software Technology, Department of Computer Science and Technology, Nanjing University, Nanjing, Jiangsu 210023, China. Tel.: +86 25 89683163; Fax: +86 25 89683163; E-mail: chjwang@nju.edu.cn.
Note: [1] This article is an extended version of the paper of Lu et al. (2017) presented at the 31st AAAI conference (San Francisco, CA USA, February 4–9, 2017).
Abstract: The volume of short text data increases rapidly these years. Data examples include tweets and online Q&A pairs. It is essential to organize and summarize these data automatically. Topic model is one of the effective approaches, whose application domains include text mining, personalized recommendation and so on. Conventional models like pLSA and LDA are designed for long text data. However, these models may suffer from the sparsity problem brought by lacking words in short text scenarios. Recent studies such as BTM show that using word co-occurrent pairs is effective to relieve the sparsity problem. However, both BTM and extended models ignore the quantifiable relationship between words. From our perspectives, two more related words should occur in the same topic. Based on this idea, we introduce a model named RIBS, which makes use of RNN to learn relationship. By using the learned relationship, we introduce a model named RIBS-Bigrams, which can display topics with bigrams. Through experiments on two open-source and real-world datasets, RIBS achieves better coherence in topic discovery, and RIBS-Bigrams achieves better readability in topic display. In the document characterization task, the document representation of RIBS can lead better purity and entropy in clustering, higher accuracy in classification.
Keywords: Topic model, short text, Recurrent Neural Network, bigrams
