Addressing challenges in multi-label text classification: A joint attention and shared semantic space approach

Article type: Research Article

Authors: Gui, Zhen^{; *}

Affiliations: Hunan International Economics University, Changsha, China

Correspondence: [*] Corresponding author. Zhen Gui. E-mail: 240670821@qq.com.

Abstract: The task of multi-label text classification involves assigning a set of related labels to a given document. However, there are three main problems with this task. Firstly, the joint modeling of label-text and label-label relationships is inadequate. Secondly, the semantic mining of the label itself is insufficient. Lastly, the utilization of the internal structure information of the label is ignored. To address these issues, a new multi-label text classification method has been proposed. This method is based on joint attention and shared semantic space. The joint multi-head attention mechanism models the relationship between labels and documents as well as the relationship between labels simultaneously. This helps to avoid error transmission and utilizes the interaction information between them. The decouple shared semantic space embedding method improves the method of using labels semantic information and reduces deviation in the phase of modeling correlation. The hierarchical hinting method based on prior knowledge relies on the prior knowledge in the pre-trained model to exploit the labels hierarchy information. Experimental results have shown that this proposed method is superior to existing multi-label text classification methods in public datasets.

Keywords: Multi-label text classification, attention mechanism, label representation, semantic embedding, pre-trained model

DOI: 10.3233/JIFS-234151

Journal: Journal of Intelligent & Fuzzy Systems, vol. 45, no. 6, pp. 10649-10659, 2023