Research on named entity recognition of chinese electronic medical records based on multi-head attention mechanism and character-word information fusion

Article type: Research Article

Authors: Zhang, Qinghui^{a; b} | Wu, Meng^a | Lv, Pengtao^{a; *} | Zhang, Mengya^a | Yang, Hongwei^a

Affiliations: [a] Key Laboratory of Grain Information Processing and Control, Ministry of Education, Henan University of Technology, Zhengzhou, Henan, PR China | [b] Henan Key Laboratory of Grain Photoelectric Detection and Control, Henan University of Technology, Zhengzhou, Henan, PR China

Correspondence: [*] Corresponding author. Pengtao Lv, Key Laboratory of Grain Information Processing and Control, Ministry of Education, Henan University of Technology, Zhengzhou, Henan, PR China 450001 E-mail: pengtaolv@163.com.

Abstract: In the medical field, Named Entity Recognition (NER) plays a crucial role in the process of information extraction through electronic medical records and medical texts. To address the problems of long distance entity, entity confusion, and difficulty in boundary division in the Chinese electronic medical record NER task, we propose a Chinese electronic medical record NER method based on the multi-head attention mechanism and character-word fusion. This method uses a new character-word joint feature representation based on the pre-training model BERT and self-constructed domain dictionary, which can accurately divide the entity boundary and solve the impact of unregistered words. Subsequently, on the basis of the BiLSTM-CRF model, a multi-head attention mechanism is introduced to learn the dependency relationship between remote entities and entity information in different semantic spaces, which effectively improves the performance of the model. Experiments show that our models have better performance and achieves significant improvement compared to baselines. The specific performance is that the F1 value on the Chinese electronic medical record data set reaches 95.22%, which is 2.67%higher than the F1 value of the baseline model.

Keywords: Chinese electronic medical records, name entity recognition, character-word information fusion, multi-head attention

DOI: 10.3233/JIFS-212495

Journal: Journal of Intelligent & Fuzzy Systems, vol. 42, no. 4, pp. 4105-4116, 2022