Coordination in multiagent reinforcement learning systems by virtual reinforcement signals

Article type: Research Article

Authors: Kamal, M.A.S.^{a; *} | Murata, Junichi^b

Affiliations: [a] Department of Mechatronics Engineering, International Islamic University Malaysia, PO Box 10, 50728 Kuala Lumpur, Malaysia | [b] Department of Electrical and Electronic Systems Engineering, Kyushu University, 744 Motooka, Nishi-Ku, Fukuoka 819-0395, Japan

Correspondence: [*] Corresponding author. Tel.: +603 6196 4417; Fax: +603 6196 4433; E-mail: maskamal@ieee.org

Abstract: This paper presents a novel method for on-line coordination in multiagent reinforcement learning systems. In this method a reinforcement-learning agent learns to select its action estimating system dynamics in terms of both the natural reward for task achievement and the virtual reward for cooperation. The virtual reward for cooperation is ascertained dynamically by a coordinating agent who estimates it from the change in degree of cooperation of all agents using a separate reinforcement learning. This technique provides adaptive coordination, requires less communication and ensures agents to be cooperative. The validity of virtual rewards for convergence in learning is verified, and the proposed method is tested on two different simulated domains to illustrate its significance. The empirical performance of the coordinated system compared to the uncoordinated system illustrates its advantages for multiagent systems.

DOI: 10.3233/KES-2007-11304

Journal: International Journal of Knowledge-based and Intelligent Engineering Systems, vol. 11, no. 3, pp. 181-191, 2007