Moment matching training for neural machine translation: An empirical study

Article type: Research Article

Authors: Nguyen, Long H. B.^{a; b; *} | Pham, Nghi T.^{a; b} | Duc, Le D. C.^{a; b} | Hoang, Cong Duy Vu^c | Dinh, Dien^{a; b}

Affiliations: [a] Faculty of Information Technology, University of Science Ho Chi Minh City, Vietnam | [b] Vietnam National University, Ho Chi Minh City, Vietnam | [c] Oracle Digital Assistant, Oracle Corporation, Melbourne, Australia

Correspondence: [*] Corresponding author. Long H. B. Nguyen, E-mail: nhblong@fit.hcmus.edu.vn.

Abstract: In recent years, Neural Machine Translation (NMT), which harnesses the power of neural networks, has achieved astonishing achievements. Despite its promise, NMT models can still not model prior external knowledge. Recent investigations have necessitated the adaptation of past expertise to both training and inference methods, resulting in translation inference issues. This paper proposes an extension of the moment matching framework that incorporates advanced prior knowledge without interfering with the inference process by using a matching mechanism between the model and empirical distributions. Our tests show that the suggested expansion outperforms the baseline and effectively over various language combinations.

Keywords: Neural machine translation, moment matching, objective function

DOI: 10.3233/JIFS-213240

Journal: Journal of Intelligent & Fuzzy Systems, vol. 43, no. 3, pp. 2633-2645, 2022