Affiliations: [a] Kalasalingam University, Krishnan Kovil, Srivilliputtur, Tamil Nadu, India | [b] Berkeley Education Alliance for Research, Singapore | [c] School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran | [d] Aurel Vlaicu University of Arad, Arad, Romania | [e] Department of Engineering, University of Sannio, Benevento, Italy
Correspondence:
[*]
Corresponding author: Sreram Balasubramaniyan, Kalasalingam University, Krishnan Kovil, Srivilliputtur, Tamil Nadu 626126, India. E-mail:cpscourse@klu.ac.in
Abstract: This paper proposes a stochastic optimal controller for networked control systems (NCS) with unknown dynamics and medium access constraints. The medium access constraint of NCS is modelled as a Markov Decision Process (MDP) that switches modes depending the channel access to the actuators. We then show that using the MDP assumption, the NCS with medium access constraint can be modelled as a Markovian jump linear system. Then a stochastic optimal controller is proposed that minimizes the quadratic cost function using Q-learning algorithm. The resulting control algorithm simultaneously optimizes the quadratic cost function and also allocates the network bandwidth judiciously by designing a scheduler. Two compensation strategies transmit zero and zero-order hold for control inputs that fail to get an access to channel are studied. The proposed controller and scheduler are illustrated using experiments on networks and simulations on an industrial four-tank system. The advantage of the proposed approach is that the optimal controller and scheduler can be designed forward-in-time for NCS with unknown dynamics. This is a departure from traditional dynamic programming based approaches that assume complete knowledge of the NCS dynamics and network constraints beforehand to solve the optimal controller problem backward-in-time.
Keywords: Networked control systems (NCSs), stochastic optimal controller, q-learning, medium access, constraints, Markov Decision Process (MDP)
