A multi-agent reinforcement learning algorithm with fuzzy approximation for Distributed Stochastic Unit Commitment

Article type: Research Article

Authors: Farzaneh, Ghorbani^a | Mohsen, Afsharchi^{a; *} | Vali, Derhami^b

Affiliations: [a] University of Zanjan, Zanjan, Iran | [b] Yazd University, Yazd, Iran

Correspondence: [*] Corresponding author. Mohsen Afsharchi, University of Zanjan, Zanjan, Iran. E-mail: afsharchi@znu.ac.ir.

Abstract: This paper proposes a novel multi-agent unit commitment model under Smart Grid (SG) environment to minimize the demand satisfaction error and production cost. This is a distributed solution applicable in non-deterministic environments with stochastic parameters intending to solve Distributed Stochastic Unit Commitment (DSUC) problem. We use multi-agent reinforcement learning (RL) in which agents learn as independent learners to cooperatively satisfy the demand profile. The learning mechanism proceeds using a reward signal, which is given based on the performance of the entire system as well as the impact of the joint action of the agents. The learning agent utilizes a novel multi-agent version of Fuzzy Least Square Policy Iteration (FLSPI) as a model-free RL algorithm to approximate Q-function. Based on this approximation, the agent makes the best decision to achieve the goals while considering the constraints governing the system. Uncertainty sources in our definition of the problem are fluctuations in the predicted demand function, random productions of clean energy generators and the possibility of accidental failure in power generators. Training for one time interval (i.e. one season or one year) consisting of several time intervals (i.e. days) can be simultaneously conducted by one trial in our method. We have conducted our experiment in two different frameworks. These frameworks are defined based on the problem complexity in terms of the number of generators, the uncertainties in the environment and the system constraints. The results show that the learning agent learns to satisfy the demand profile as well as other constrains.

Keywords: Multi-agent reinforcement learning, Stochastic Unit Commitment, fuzzy approximation

DOI: 10.3233/JIFS-182879

Journal: Journal of Intelligent & Fuzzy Systems, vol. 37, no. 5, pp. 6613-6628, 2019