Affiliations: [a] School of Electrical and Information Engineering, Knowledge Based Intelligent Engineering Systems Centre, University of South Australia, Mawson Lakes SA 5095, Australia | [b] Applied Computational Intelligence Research Unit, University of the West of Scotland
Abstract: Multi-Agent Systems extend research in Artificial Intelligence and agent systems by incorporating cooperative learning and agent teaming architectures. Agent teaming is a key research area of multi-agent systems that is mainly composed of artificial intelligence and distributed computing techniques. The reasoning and learning ability of agents in uncertain environments via communication and collaboration (in both competitive and cooperative situations) is another key feature for autonomous agents. Many theoretical and applied techniques have been applied to the investigation of autonomous agents with respect to coordination, cooperation, and learning abilities. Due to the inherent complexity of real-time, stochastic, and dynamic environments, it is often extremely complex and difficult to formally verify their properties a priori. In addition, it is quite difficult to generate enough episodes via real applications for training the goal-oriented agent’s individual and cooperative learning abilities. In most cases, such abilities can be obtained via computer simulation, rather than directly from real applications. In doing so, a simulation testbed is applied to test the learning algorithms in the specified scenarios. The objective of this paper is to improve the convergence and efficiency of reinforcement learning algorithms for large, continuous state-action spaces, by finding the optimal values of the parameters for those algorithms. In this paper, the game of soccer is adopted as the simulation environment in conjunction with optimisation techniques to verify goal-oriented agents’ competitive and cooperative learning abilities for decision making. We use the Sarsa learning algorithm with a linear function approximation technique known as tile coding to avoid the state space growing exponentially. The convergence and efficiency of Sarsa algorithm are investigated through simulating a soccer game.
