Affiliations: [a] Department of Mathematical Sciences, Georgia Southern University, Statesboro, GA, USA | [b] Center for Statistical Research and Methodology, U.S. Census Bureau, Washington, DC, USA
Correspondence:
[*]
Corresponding author: Stephen W. Carden, Department of Mathematical Sciences, Box 8093, Georgia Southern University, Statesboro, GA 30460, USA. E-mail:scarden@georgiasouthern.edu
Note: [1] This report is released to inform interested parties of research and to encourage discussion. The views expressed on statistical issues are those of the author and not necessarily those of the U.S. Census Bureau.
Abstract: Reinforcement learning (RL) concerns algorithms tasked with learning optimal control
policies by interacting with or observing a system. In computer science and other fields
in which RL originated, large sample sizes are the norm, because data can be generated at will
from a generative model. Recently, RL methods have been adapted for use in clinical trials,
resulting in much smaller sample sizes. Nonparametric methods are common in RL, but are likely
to over-generalize when limited data is available. This paper proposes a novel methodology
for learning optimal policies by leveraging the researcher's partial knowledge about the
probability transition structure into an approximate generative model from which synthetic data
can be produced. Our method is applied to a scenario where the researcher must create a medical
prescription policy for managing a disease with sporadically appearing symptoms.
