Deep Learning and Hierarchical Reinforcement Learning for modeling a Conversational Recommender System

Issue title: Selected papers from the 16th International Conference of the Italian Association for Artificial Intelligence

Guest editors: Stefano Ferilli and Francesca Alessandra Lisi

Article type: Research Article

Authors: Basile, Pierpaolo^{a; *} | Greco, Claudio^a | Suglia, Alessandro^b | Semeraro, Giovanni^a

Affiliations: [a] Department of Computer Science, University of Bari Aldo Moro, Italy IT | [b] Interaction Lab, Heriot-Watt University, Edinburgh Centre for Robotics Edinburgh, Scotland, UK

Correspondence: [*] Corresponding author: Pierpaolo Basile, Department of Computer Science, University of Bari Aldo Moro, Italy IT. E-mail: pierpaolo.basile@uniba.it.

Abstract: In this paper, we propose a framework based on Hierarchical Reinforcement Learning for dialogue management in a Conversational Recommender System scenario. The framework splits the dialogue into more manageable tasks whose achievement corresponds to goals of the dialogue with the user. The framework consists of a meta-controller, which receives the user utterance and understands which goal should pursue, and a controller, which exploits a goal-specific representation to generate an answer composed by a sequence of tokens. The modules are trained using a two-stage strategy based on a preliminary Supervised Learning stage and a successive Reinforcement Learning stage.

Keywords: Conversational Recommender Systems, Deep Learning, Hierarchical Reinforcement Learning, Conversational Agents

DOI: 10.3233/IA-170031

Journal: Intelligenza Artificiale, vol. 12, no. 2, pp. 125-141, 2018