SE4SA: a deep syntactical contextualized text representation learning approach for sentiment analysis

Article type: Research Article

Authors: Vo, Tham^{; *}

Affiliations: Thu Dau Mot University, Binh Duong, Vietnam

Correspondence: [*] Corresponding author. Tham Vo, Thu Dau Mot University, Binh Duong, Vietnam. E-mail: thamvth@tdmu.edu.vn.

Abstract: Recently, many pre-trained text embedding models have been applied to effectively extract latent features from texts and achieve remarkable performance in various downstream tasks of sentiment analysis domain. However, these pre-trained text embedding models also encounter limitations related to the capability preserving the syntactical structure as well as the global long-range dependent relationships of words. Thus, they might fail to recognize the relevant syntactical features of words as valuable evidences for analyzing sentiment aspects. To overcome these limitations, we proposed a novel deep semantic contextual embedding technique for sentiment analysis, called as: SE4SA. Our proposed SE4SA is a multi-level text embedding model which enables to jointly exploit the long-range syntactical and sequential representations of texts. Then, these achieved rich semantic textual representations can support to have a better understanding on the sentiment aspects of the given text corpus, thereby resulting the better performance on sentiment analysis task. Extensive experiments in several benchmark datasets demonstrate the effectiveness or our proposed SE4SA model in comparing with recent state-of-the-art model.

Keywords: Sentiment analysis, GCN, BERT, attention, masked language model

DOI: 10.3233/JIFS-211535

Journal: Journal of Intelligent & Fuzzy Systems, vol. 41, no. 6, pp. 7527-7546, 2021