Smart sampling for lightweight verification of Markov decision processes

被引：31

作者：

D'Argenio, Pedro ^{[1
]}

Legay, Axel ^{[2
]}

Sedwards, Sean ^{[2
]}

Traonouez, Louis-Marie ^{[2
]}

机构：

[1] Univ Nacl Cordoba, RA-5000 Cordoba, Argentina

[2] Inria Rennes Bretagne Atlantique, Rennes, France

来源：

INTERNATIONAL JOURNAL ON SOFTWARE TOOLS FOR TECHNOLOGY TRANSFER | 2015年 / 17卷 / 04期

关键词：

Statistical model checking; Sampling; Nondeterminism; REAL APPLICATIONS; INEQUALITIES;

D O I：

10.1007/s10009-015-0383-0

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

Markov decision processes (MDP) are useful to model optimisation problems in concurrent systems. To verify MDPs with efficient Monte Carlo techniques requires that their nondeterminism be resolved by a scheduler. Recent work has introduced the elements of lightweight techniques to sample directly from scheduler space, but finding optimal schedulers by simple sampling may be inefficient. Here we describe "smart" sampling algorithms that can make substantial improvements in performance.

引用

页码：469 / 484

页数：16

共 50 条

[41] Possibilistic Markov decision processes
Sabbadin, R
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2001, 14 (03) : 287 - 300
[42] On constrained Markov decision processes
Department of Econometrics, University of Sydney, Sydney, NSW 2006, Australia
不详
Oper Res Lett, 1 (25-28):
[43] Robust Markov Decision Processes
Wiesemann, Wolfram
Kuhn, Daniel
Rustem, Berc
MATHEMATICS OF OPERATIONS RESEARCH, 2013, 38 (01) : 153 - 183
[44] On the detection of Markov decision processes
Duan, Xiaoming
Savas, Yagiz
Yan, Rui
Xu, Zhe
Topcu, Ufuk
AUTOMATICA, 2025, 175
[45] Information Directed Policy Sampling for Partially Observable Markov Decision Processes with Parametric Uncertainty
Kumar, Peeyush
Ghate, Archis
ADVANCES IN SERVICE SCIENCE, 2019, : 201 - 209
[46] A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes
Michael Kearns
Yishay Mansour
Andrew Y. Ng
Machine Learning, 2002, 49 : 193 - 208
[47] A sparse sampling algorithm for near-optimal planning in large Markov decision processes
Kearns, M
Mansour, Y
Ng, AY
MACHINE LEARNING, 2002, 49 (2-3) : 193 - 208
[48] A sparse sampling algorithm for near-optimal planning in large Markov decision processes
Kearns, M
Mansour, Y
Ng, AY
IJCAI-99: PROCEEDINGS OF THE SIXTEENTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOLS 1 & 2, 1999, : 1324 - 1331
[49] Ordinal Decision Models for Markov Decision Processes
Weng, Paul
20TH EUROPEAN CONFERENCE ON ARTIFICIAL INTELLIGENCE (ECAI 2012), 2012, 242 : 828 - 833
[50] Markov Decision Processes with Arbitrary Reward Processes
Yu, Jia Yuan
Mannor, Shie
Shimkin, Nahum
MATHEMATICS OF OPERATIONS RESEARCH, 2009, 34 (03) : 737 - 757

← 1 2 3 4 5 →