Correlation alignment with attention mechanism for unsupervised domain adaptation

Article type: Research Article

Authors: Chen, Rong | Ren, Chongguang^{; *}

Affiliations: College of Computer Science and Technology, Shandong University of Technology, China. E-mail: ipaperren@163.com

Correspondence: [*] Corresponding author. E-mail: rongcx@foxmail.com.

Abstract: Domain adaptation aims to solve the problems of lacking labels. Most existing works of domain adaptation mainly focus on aligning the feature distributions between the source and target domain. However, in the field of Natural Language Processing, some of the words in different domains convey different sentiment. Thus not all features of the source domain should be transferred, and it would cause negative transfer when aligning the untransferable features. To address this issue, we propose a Correlation Alignment with Attention mechanism for unsupervised Domain Adaptation (CAADA) model. In the model, an attention mechanism is introduced into the transfer process for domain adaptation, which can capture the positively transferable features in source and target domain. Moreover, the CORrelation ALignment (CORAL) loss is utilized to minimize the domain discrepancy by aligning the second-order statistics of the positively transferable features extracted by the attention mechanism. Extensive experiments on the Amazon review dataset demonstrate the effectiveness of CAADA method.

Keywords: Domain adaptation, CORAL, attention mechanism, domain discrepancy

DOI: 10.3233/WEB-210447

Journal: Web Intelligence, vol. 18, no. 4, pp. 261-267, 2020