Efficient Exploration in Reinforcement Learning Based on Utile Suffix Memory

Article type: Research Article

Authors: Pchelkin, Arthur

Affiliations: Faculty of Computer Science and Information Technology, Riga Technical University, 1 Kalku Str., LV‐1658 Riga, Latvia, e‐mail: arturp@balticom.lv

Abstract: Reinforcement learning addresses the question of how an autonomous agent can learn to choose optimal actions to achieve its goals. Efficient exploration is of fundamental importance for autonomous agents that learn to act. Previous approaches to exploration in reinforcement learning usually address exploration in the case when the environment is fully observable. In contrast, we study the case when the environment is only partially observable. We consider different exploration techniques applied to the learning algorithm “Utile Suffix Memory”, and, in addition, discuss an adaptive fringe depth. Experimental results in a partially observable maze show that exploration techniques have serious impact on performance of learning algorithm.

Keywords: reinforcement learning, exploration, hidden state, short‐term memory

Journal: Informatica, vol. 14, no. 2, pp. 237-250, 2003