Average cost Markov control processes: stability with respect to the Kantorovich metric

被引：6

作者：

Gordienko, Evgueni ^{[1
]}

Lemus-Rodriguez, Enrique ^{[2
]}

Montes-de-Oca, Raul ^{[1
]}

机构：

[1] Univ Autonoma Metropolitana, Unidad Iztapalapa, Mexico City 09340, DF, Mexico

[2] Univ Anahuac, Colonia Lomas Anahuac Hu 52786, Edo De Mexico, Mexico

来源：

MATHEMATICAL METHODS OF OPERATIONS RESEARCH | 2009年 / 70卷 / 01期

关键词：

Discrete-time Markov control process; Average cost; Contraction; Stability inequality; Kantorovich metric;

D O I：

10.1007/s00186-008-0229-6

中图分类号：

C93 [管理学]; O22 [运筹学];

学科分类号：

070105 ; 12 ; 1201 ; 1202 ; 120202 ;

摘要：

We study perturbations of a discrete-time Markov control process on a general state space. The amount of perturbation is measured by means of the Kantorovich distance. We assume that an average (per unit of time on the infinite horizon) optimal control policy can be found for the perturbed (supposedly known) process, and that it is used to control the original (unperturbed) process. The one-stage cost is not assumed to be bounded. Under Lyapunov-like conditions we find upper bounds for the average cost excess when such an approximation is used in place of the optimal (unknown) control policy. As an application of the found inequalities we consider the approximation by relevant empirical distributions. We illustrate our results by estimating the stability of a simple autoregressive control process. Also examples of unstable processes are provided.

引用

页码：13 / 33

页数：21

共 50 条

[31] Time and ratio expected average cost optimality for semi-Markov control processes on borel spaces
Luque-Vásquez, F
Vega-Amaya, O
COMMUNICATIONS IN STATISTICS-THEORY AND METHODS, 2004, 33 (03) : 715 - 734
[32] Bayesian adaptive control of discrete-time Markov processes with long-run average cost
Dipto. di Matemat. Pura ed Applicata, Universitá di Padova, LADSEB - CNR, Via Belzoni 7, I 35131 Padova, Italy
不详
Syst Control Lett, 1-2 (55-62):
[33] Bayesian adaptive control of discrete-time Markov processes with long-run average cost
Di Masi, GB
Stettner, L
SYSTEMS & CONTROL LETTERS, 1998, 34 (1-2) : 55 - 62
[34] AVERAGE COST OPTIMAL POLICIES FOR MARKOV CONTROL PROCESSES WITH BOREL STATE-SPACE AND UNBOUNDED COSTS
HERNANDEZLERMA, O
LASSERRE, JB
SYSTEMS & CONTROL LETTERS, 1990, 15 (04) : 349 - 356
[35] Markov Control Processes with the Expected Total Cost Criterion: Optimality, Stability, and Transient Models
Onésimo Hernández-Lerma
Guadalupe Carrasco
Rubén Pérez-Hernández
Acta Applicandae Mathematica, 1999, 59 : 229 - 269
[36] Markov control processes with the expected total cost criterion:: Optimality, stability, and transient models
Hernández-Lerma, O
Carrasco, G
Pérez-Hernández, R
ACTA APPLICANDAE MATHEMATICAE, 1999, 59 (03) : 229 - 269
[37] Markov control processes with randomized discounted cost
Juan González-Hernández
Raquiel R. López-Martínez
J. Rubén Pérez-Hernández
Mathematical Methods of Operations Research, 2007, 65 : 27 - 44
[38] Markov control processes with randomized discounted cost
Gonzalez-Hernandez, Juan
Lopez-Martinez, Raquiel R.
Ruben Perez-Hernandez, J.
MATHEMATICAL METHODS OF OPERATIONS RESEARCH, 2007, 65 (01) : 27 - 44
[39] ON PARTIALLY OBSERVABLE MARKOV DECISION-PROCESSES WITH AN AVERAGE COST CRITERION
FERNANDEZGAUCHERAND, E
ARAPOSTATHIS, A
MARCUS, SI
PROCEEDINGS OF THE 28TH IEEE CONFERENCE ON DECISION AND CONTROL, VOLS 1-3, 1989, : 1267 - 1273
[40] Reinforcement learning based algorithms for average cost Markov Decision Processes
Abdulla, Mohammed Shahid
Bhatnagar, Shalabh
DISCRETE EVENT DYNAMIC SYSTEMS-THEORY AND APPLICATIONS, 2007, 17 (01): : 23 - 52

← 1 2 3 4 5 →