RankNet for evaluation functions of the game of Go

Article type: Research Article

Authors: Mandai, Yusaku^{; *} | Kaneko, Tomoyuki

Affiliations: University of Tokyo, Tokyo, Japan

Correspondence: [*] Corresponding author. E-mail: mandai@graco.c.u-tokyo.ac.jp.

Abstract: In this paper, we present a new algorithm for learning evaluation functions of the game of Go. Recently AlphaGo Zero and AlphaZero have shown that accurate evaluation functions can be constructed by using deep neural networks. Such a training, however, requires an enormous amount of computational resources that are not available for most researchers. One of the next challenges in this domain is constructing accurate evaluation functions with lesser computational resources. To tackle this problem, we apply the RankNet algorithm to training an AlphaGo Zero style unified Policy and Value network in a learning-to-rank fashion. Using the pairwise RankNet training increases the potential number of training examples and alleviates the requirements for the number of game records. Our modified RankNet algorithm trains both value and policy losses and its joint training makes the learning stable. Experimental results showed that neural networks trained by our algorithm showed higher playing strength than other methods, especially when the dataset sizes were relatively limited.

DOI: 10.3233/ICG-190108

Journal: ICGA Journal, vol. 41, no. 2, pp. 78-91, 2019