Affiliations: [a] Department of Computer Science, National Chiao Tung University, Hsinchu, Taiwan | [b] Institute of Information Science, Academia Sinica, Taipei, Taiwan | [c] National Taipei University, Taiwan
Note: [**] Department of Computer Science and Information Engineering, National Taipei University, New Taipei City, Taiwan.
Abstract: For developers of game-playing programs, it is an important but non-trivial task to measure the strengths of the programs, especially with respect to the theoretically optimal players. The optimal players are hard to obtain in practice since many real-world games are too hard to solve entirely. In this paper, a reduced version of Chinese dark chess (CDC), 2×4 CDC, which has been solved, is used as a testbed to analyze the strengths of game-playing programs for CDC, a non-deterministic game. Absolute strength is then defined to be the win rates against the optimal player. Experiment results show that win rates against designated baseline programs are a good metric, since these win rates are highly correlated to absolute strengths. The stronger the designated baseline programs are, the higher the correlations of win rates are to the absolute strengths. In addition, it is shown that prediction rates and mean squared errors are other good metrics and have high correlations to the absolute strengths. The experiment results also show that the win rates obtained in 2×4 CDC have a high correlation to those in the original 4×8 CDC, except for those using the technique of progressive bias.
Keywords: Chinese dark chess, strength analysis, optimal player, game-playing program
DOI: 10.3233/ICG-180046
Journal: ICGA Journal, vol. 40, no. 2, pp. 77-104, 2018
