Barycentric quantization for planning in continuous domains

Article type: Research Article

Authors: Nikovski, Daniel Nikolaev

Affiliations: Mitsubishi Electric Research Labs, 201 Broadway, Cambridge, USA. E-mail: nikovski@merl.com

Abstract: We consider the class of planning and sequential decision making problems where the state space has continuous components, but the available actions come from a discrete set, and argue that a suitable approach for solving them could involve an appropriate quantization scheme for the continuous state variables, followed by approximate dynamic programming. We propose one such scheme based on barycentric approximations that effectively converts the continuous dynamics into a Markov decision process, and demonstrate that it can be viewed both as an approximation to the continuous dynamics, as well as a value function approximator over the continuous domain. We describe the application of this method to several hard industrial problems, and point out additional candidate problems that could be amenable to it.

Keywords: Planning, Markov decision processes

DOI: 10.3233/AIC-150675

Journal: AI Communications, vol. 28, no. 3, pp. 539-551, 2015