Searching for just a few words should be enough to get started. If you need to make more complex queries, use the tips below to guide you.
Article type: Research Article
Authors: Demura, Yosuke | Kaneko, Tomoyuki; *
Affiliations: Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan
Correspondence: [*] Corresponding author. E-mail: kaneko@acm.org.
Abstract: AlphaZero has achieved superhuman performance in Go, chess, and shogi with a general reinforcement learning (RL) algorithm. This achievement is remarkable because AlphaZero does not rely on any training dataset of strong players. However, AlphaZero-style training requires substantial computational resources. Gumbel AlphaZero, a recently introduced more efficient version of AlphaZero, reduces the computational cost of AlphaZero training. The goal of this study is to further improve the playing strength of Gumbel AlphaZero under a limited amount of computational resources. We focus on the diversity in training games, inspired by procedural generation and domain randomization in RL studies, and propose a novel method, initial state diversification. This method diversifies the initial states of a self-play game to encourage the RL agent to understand the game in a more general manner through diverse experiences. For example, in shogi, the initial state of each self-play game is diversified by rearranging the pieces under realistic domain constraints. Experiments demonstrated that training with initial state diversification improves the playing strength of Gumbel AlphaZero in shogi, within the same computational budget for training.
Keywords: Reinforcement learning, AlphaZero, Gumbel AlphaZero, shogi, Chess960
DOI: 10.3233/ICG-240255
Journal: ICGA Journal, vol. Pre-press, no. Pre-press, pp. 1-27, 2024
IOS Press, Inc.
6751 Tepper Drive
Clifton, VA 20124
USA
Tel: +1 703 830 6300
Fax: +1 703 830 2300
sales@iospress.com
For editorial issues, like the status of your submitted paper or proposals, write to editorial@iospress.nl
IOS Press
Nieuwe Hemweg 6B
1013 BG Amsterdam
The Netherlands
Tel: +31 20 688 3355
Fax: +31 20 687 0091
info@iospress.nl
For editorial issues, permissions, book requests, submissions and proceedings, contact the Amsterdam office info@iospress.nl
Inspirees International (China Office)
Ciyunsi Beili 207(CapitaLand), Bld 1, 7-901
100025, Beijing
China
Free service line: 400 661 8717
Fax: +86 10 8446 7947
china@iospress.cn
For editorial issues, like the status of your submitted paper or proposals, write to editorial@iospress.nl
如果您在出版方面需要帮助或有任何建, 件至: editorial@iospress.nl