Affiliations: [a] IITiS, Polish Academy of Sciences, ul. Bałtycka 5, 44-100 Gliwice, Poland. arturrataj@gmail.com | [b] IMCS, Jan Długosz University, Al. Armii Krajowej 13/15, 42-200 Częstochowa, Poland. b.wozna@ajd.czest.pl
Correspondence:
[*]
Address for correspondence: IITiS, Polish Academy of Sciences, ul. Bałtycka 5, 44-100 Gliwice, Poland
Abstract: We present different ways of an approximate extrapolation of an optimal policy of a small model to that of a large equivalent of the model, which itself is too large to find its exact policy directly using probabilistic model checking (PMC). In particular, we obtain a global optimal resolution of non-Cdeterminism in several small Markov Decision Processes (MDP) or its extensions like Stochastic Multi-player Games (SMG) using PMC. We then use that resolution to form a hypothesis about an analytic decision boundary representing a respective policy in an equivalent large MDP/SMG. The resulting hypothetical decision boundary is then statistically approximately verified, if it is locally optimal and if it indeed represents a “good enough” policy. The verification either weakens or strengthens the hypothesis. The criterion of the optimality of the policy can be expressed in any modal logic that includes a version of the probabilistic operator P~p[·], and for which a PMC method exists.
Keywords: probabilistic model checking, statistical model checking, non-determinism, optimal policy, extrapolation
