Affiliations: [a] Department of Electronic Engineering, Tsinghua University, Beijing, China | [b] Tsinghua-iFlytek Joint Laboratory for Speech Technology, Beijing, China
Correspondence:
[*]
Corresponding author: Zhiyang He, Department of Electronic Engineering, Tsinghua University, Beijing, China. E-mail: zyhe_ts@mail.tsinghua.edu.cn.
Abstract: In the current paper, we propose a probabilistic generative model, the label correlation mixture model (LCMM), to depict multi-labeled document data, which can be utilized for multi-label text classification. LCMM assumes two stochastic generative processes, which correspond to two submodels: 1) a label correlation model; and 2) a label mixture model. The former model formulates labels’ generative process, in which a label correlation network is created to depict the dependency between labels. Moreover, we present an efficient inference algorithm for calculating the generative probability of a multi-label class. Furthermore, in order to optimize the label correlation network, we propose a parameter-learning algorithm based on gradient descent. The second submodel in the LCMM depicts the generative process of words in a document with the given labels. Different traditional mixture models can be adopted in this generative process, such as the mixture of language models, or topic models. In the multi-label classification stage, we propose a two-step strategy to most efficiently utilize the LCMM based on the framework of Bayes decision theory. We conduct extensive multi-label classification experiments on three standard text data sets. The experimental results show significant performance improvements comparing to existing approaches. For example, the improvements on accuracy and macro F-score measures in the OHSUMED data set achieve 28.3% and 37.0%, respectively. These performance enhancements demonstrate the effectiveness of the proposed models and solutions.
Keywords: Label correlation mixture model, probabilistic generative model, multi-label text classification, label correlation model, label correlation network, Bayes decision theory
