Affiliations: [a] Faculty of Computer Science and Engineering, Shahid Beheshti University, Tehran, Iran | [b] School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran | [c] Institute of Informatics, University of Federal Rio Grand do Sul, Porto Alegre, Brazil
Correspondence:
[*]
Corresponding author: Monireh Abdoos, School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran. E-mail: abdoos@iust.ac.ir.
Abstract: Holonic multi-agent system (HMAS) offers a promising approach to model complex systems. HMAS is based on self-similar entities defined in a holarchical organization. Although some models and frameworks have been proposed for holonic systems, there is no general reinforcement learning method that can be easily implemented in HMAS. This paper presents a reinforcement learning method for HMAS. The holons in different levels have direct effect on learning process of each other through communication. For hierarchical communications between holons, abstract data flows are defined that are used for state estimation, action selection and reward calculation. The proposed learning method includes a self-similar structure in which the learning processes of the holons are independent of their actual positions in the holarchy. A real-world application is also used to show that how the holons can be implemented in practice. Experimental results show that the proposed holonic reinforcement learning method improves the performance.
Keywords: Holonic multi-agent systems, reinforcement learning, abstract data flow
