Similarity of learned helplessness in human being and fuzzy reinforcement learning algorithms

Issue title: Computational intelligence models for image processing and information reasoning

Article type: Research Article

Authors: Derhami, Vali

Affiliations: Electrical and Computer Engineering Department, Yazd University, Yazd, Iran

Note: [] Corresponding author. Vali Derhami, Electrical and Computer Engineering Department, Yazd University, Yazd, Iran. Tel.: +98 351 8122356; Fax: +98 351 8122357; E-mail: vderhami@yazduni.ac.ir

Abstract: The sequential and uncontrolled punishments in social life may lead to what psychologists call learned helplessness or depression. Like learning in social life, agents based on Fuzzy Reinforcement Learning (FRL) sometimes cannot learn well. Experiments show that if an agent continuously performs actions that cause sequential punishments in the beginning of learning, then it does not usually behave well and often selects actions that evoke punishments. Therefore, the learning takes so long or not successful. In this paper, we address this issue called faulty learning in RL algorithm by exploiting learned helplessness. We demonstrate learned helplessness in the training of an agent by FRL algorithm and analyze it. The result of analysis shows that since the action value function is approximated by a fuzzy system; continuous punishments lead all weight parameters of the approximator toward negative amounts. Hence, the agent cannot learn well. To prevent this problem, we propose a new reinforcement function. The proposed reinforcement function is adaptive and depends on the number of visit of the state. Simulation results show that new reinforcement function prevents learned helplessness and improves the learning in terms of learning speed and action quality. The proposed ideas can be used and extended to our social and psychology life.

Keywords: Learned helplessness, fuzzy systems, reinforcement learning, Sarsa

DOI: 10.3233/IFS-2012-0558

Journal: Journal of Intelligent & Fuzzy Systems, vol. 24, no. 2, pp. 347-354, 2013