The complexity of reachability in parametric Markov decision processes

被引：11

作者：

Junges, Sebastian ^{[1
]}

Katoen, Joost-Pieter ^{[2
]}

Perez, Guillermo A. ^{[3
]}

Winkler, Tobias ^{[2
]}

机构：

[1] Univ Calif Berkeley, Berkeley, CA 94720 USA

[2] Rhein Westfal TH Aachen, Aachen, Germany

[3] Univ Antwerp, Antwerp, Belgium

来源：

JOURNAL OF COMPUTER AND SYSTEM SCIENCES | 2021年 / 119卷

基金：

欧洲研究理事会;

关键词：

Parametric Markov decision processes; Formal verification; Existential theory of the reals; Computational complexity; Parameter synthesis; SYSTEMS; MODELS;

D O I：

10.1016/j.jcss.2021.02.006

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

This article presents the complexity of reachability decision problems for parametric Markov decision processes (pMDPs), an extension to Markov decision processes (MDPs) where transitions probabilities are described by polynomials over a finite set of parameters. In particular, we study the complexity of finding values for these parameters such that the induced MDP satisfies some maximal or minimal reachability probability constraints. We discuss different variants depending on the comparison operator in the constraints and the domain of the parameter values. We improve all known lower bounds for this problem, and notably provide ETR-completeness results for distinct variants of this problem. (C) 2021 Elsevier Inc. All rights reserved.

引用

页码：183 / 210

页数：28

共 50 条

[31] Solving Markov Decision Processes with Reachability Characterization from Mean First Passage Times
Debnath, Shoubhik
Liu, Lantao
Sukhatme, Gaurav
2018 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2018, : 7063 - 7070
[32] THE COMPLEXITY OF REACHABILITY IN DISTRIBUTED COMMUNICATING PROCESSES
REIF, JH
SMOLKA, SA
ACTA INFORMATICA, 1988, 25 (03) : 333 - 354
[33] Reachability in Parametric Interval Markov Chains Using Constraints
Bart, Anicet
Delahaye, Benoit
Lime, Didier
Monfroy, Eric
Truchet, Charlotte
QUANTITATIVE EVALUATION OF SYSTEMS (QEST 2017), 2017, 10503 : 173 - 189
[34] Reachability in parametric Interval Markov Chains using constraints
Bart, Anicet
Delahaye, Benoit
Fournier, Paulin
Lime, Didher
Monfroy, Eric
Truchet, Charlotte
THEORETICAL COMPUTER SCIENCE, 2018, 747 : 48 - 74
[35] Reachability questions in piecewise deterministic Markov processes
Bujorianu, ML
Lygeros, J
HYBRID SYSTEMS: COMPUTATION AND CONTROL, PROCEEDINGS, 2003, 2623 : 126 - 140
[36] Optimization of Parametric Policies of Markov Decision Processes under a Variance Criterion
Xia, Li
2016 13TH INTERNATIONAL WORKSHOP ON DISCRETE EVENT SYSTEMS (WODES), 2016, : 332 - 337
[37] Planning in Markov Decision Processes with Gap-Dependent Sample Complexity
Jonsson, Anders
Kaufmann, Emilie
Menard, Pierre
Domingues, Omar Darwiche
Leurent, Edouard
Valko, Michal
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020, 2020, 33
[38] Reducing Computational Complexity in Markov Decision Processes using Abstract Actions
Garcia-Hernandez, Ma. de Guadalupe
Ruiz-Pinales, Jose
Reyes-Ballesteros, Alberto
Onaindia, Eva
Avina-Cervantes, J. Gabriel
PROCEEDINGS OF THE SPECIAL SESSION OF THE SEVENTH MEXICAN INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE - MICAI 2008, 2008, : 256 - +
[39] Maximal Cost-Bounded Reachability Probability on Continuous-Time Markov Decision Processes
Fu, Hongfei
FOUNDATIONS OF SOFTWARE SCIENCE AND COMPUTATION STRUCTURES, 2014, 8412 : 73 - 87
[40] Markov decision processes
White, D.J.
Journal of the Operational Research Society, 1995, 46 (06):

← 1 2 3 4 5 →