Affiliations: [a] School of Artificial Intelligence, Jianghan University, Hubei, China | [b] School of Computer Science and Technology, Wuhan University of Technology, Hubei, China
Correspondence:
[*]
Corresponding author: Wang Gao, School of Artificial Intelligence, Jianghan University, Hubei, China. E-mail: gaowang2000@foxmail.com.
Abstract: Harmful information identification is a critical research topic in natural language processing. Existing approaches have been focused either on rule-based methods or harmful text identification of normal documents. In this paper, we propose a BERT-based model to identify harmful information from social media, called Topic-BERT. Firstly, Topic-BERT utilizes BERT to take additional information as input to alleviate the sparseness of short texts. The GPU-DMM topic model is used to capture hidden topics of short texts for attention weight calculation. Secondly, the proposed model divides harmful short text identification into two stages, and different granularity labels are identified by two similar sub-models. Finally, we conduct extensive experiments on a real-world social media dataset to evaluate our model. Experimental results demonstrate that our model can significantly improve the classification performance compared with baseline methods.
Keywords: Harmful short text, text classification, BERT, topic model, GPU-DMM
