Affiliations: [a]
Institute of Geospatial Information, Information Engineering University, Henan, Zhengzhou, China
| [b]
School of Geography and Tourism, Zhengzhou Normal University, Henan, Zhengzhou, China
Correspondence:
[*]
Corresponding author. Nan Jang, Institute of Geospatial Information, Information Engineering University. Henan, Zhengzhou, 450001, China. E-mail: 13653802609@163.com.
Note: [1] Foundation item: National Key Research and Development
Program of China, No. 2021YFB3900900.
Abstract: Agent-based combat simulation is an important research method in the field of military science and system simulation. Behaviour decision model plays the key role in the design of combat simulation agents. The behaviour tree (BT) designed by nonplayer characters (NPCs) in the game provides an efficient and concise method for the construction of combat simulation agents and has been widely used. Because the rationality of BT construction directly affects the rationality of agent decision logic, designing a reasonable BT has become a crucial step. The design of the operational agent BT not only relies on rich tactical experience but also needs to repeatedly adjust and optimize the BT according to the operational deduction and simulation results. To avoid unreasonable BT design caused by lack of experience and eliminate the process of repeated debugging, a modelling method of a combat simulation agent that combines reinforcement learning and the BT method was proposed. This method not only makes the design process of BT more automatic but also simplifies the experience requirements of the combat simulation agent designers. Experiments show that RL-BT effectively integrates the reinforcement learning method and can endow combat simulation agents with battlefield learning ability while making independent decisions. The agent based on RL-BT for decision modelling can continuously adjust and optimize the decision process through experience accumulation, and its performance in combat simulation is significantly better than that of the agent using the original BT.
