Affiliations: [a] Department of Mechanical Engineering, Sharif University of Technology, Tehran, Iran | [b] Department of Electrical and Computer Engineering, The University of Tehran, Tehran, Iran
Correspondence:
[*]
Address for correspondence: Prof. M. Parnianpour, Department of Mechanical Engineering, Sharif University of Technology, P.O. Box 11365-9567. Azadi Ave., Tehran, Iran. Tel.: +98/21 600 5716, 602 2707, 602 2708; Fax: +98/21 600 0021; E-mail: parnianpour@sharif.edu.
Abstract: In this study, we consider the role of multisensor data fusion in neuromuscular control using an actor-critic reinforcement learning method. The model we use is a single link system actuated by a pair of muscles that are excited with alpha and gamma signals. Various physiological sensor information such as proprioception, spindle sensors, and Golgi tendon organs have been integrated to achieve an oscillatory movement with variable amplitude and frequency, while achieving a stable movement with minimum metabolic cost and coactivation. The system is highly nonlinear in all its physical and physiological attributes. Transmission delays are included in the afferent and efferent neural paths to account for a more accurate representation of the reflex loops. This paper proposes a reinforcement learning method with an Actor-Critic architecture instead of middle and low level of central nervous system (CNS). The Actor in this structure is a two layer feedforward neural network and the Critic is a model of the cerebellum. The Critic is trained by the State-Action-Reward-State-Action (SARSA) method. The Critic will train the Actor by supervisory learning based on previous experiences. The reinforcement signal in SARSA is evaluated based on available alternatives concerning the concept of multisensor data fusion. The effectiveness and the biological plausibility of the present model are demonstrated by several simulations. The system showed excellent tracking capability when we integrated the available sensor information. Addition of a penalty for activation of muscles resulted in much lower muscle coactivation while keeping the movement stable.
Keywords: motor control, reinforcement learning, multisensor data fusion, Actor-Critic, CMAC, simulink
