A Version of the Euler Equation in Discounted Markov Decision Processes

被引：2

作者：

Cruz-Suarez, H. ^{[1
]}

Zacarias-Espinoza, G. ^{[1
]}

Vazquez-Guevara, V. ^{[1
]}

机构：

[1] Benemerita Univ Autonoma Puebla, Fac Ciencias Fis Matemat, CU, Puebla 72570, PUE, Mexico

来源：

JOURNAL OF APPLIED MATHEMATICS | 2012年

关键词：

UNCERTAINTY; GROWTH;

D O I：

10.1155/2012/103698

中图分类号：

O29 [应用数学];

学科分类号：

070104 ;

摘要：

This paper deals with Markov decision processes (MDPs) on Euclidean spaces with an infinite horizon. An approach to study this kind of MDPs is using the dynamic programming technique (DP). Then the optimal value function is characterized through the value iteration functions. The paper provides conditions that guarantee the convergence of maximizers of the value iteration functions to the optimal policy. Then, using the Euler equation and an envelope formula, the optimal solution of the optimal control problem is obtained. Finally, this theory is applied to a linear-quadratic control problem in order to find its optimal policy.

引用

页数：16

共 50 条

[31] Deterministic Discounted Markov Decision Processes with Fuzzy Rewards/Costs
Cruz-Suarez, Hugo
Montes-de-Oca, Raul
Ortega-Gutierrez, R. Israel
FUZZY INFORMATION AND ENGINEERING, 2023, 15 (03) : 274 - 290
[32] Primal-Dual Algorithms for Discounted Markov Decision Processes
Cogill, Randy
2015 EUROPEAN CONTROL CONFERENCE (ECC), 2015, : 260 - 265
[33] Homotopy approach for infinite horizon discounted Markov decision processes
White, Douglas John
ZOR. Zeitschrift fur Operations-Research, 1996, 43 (03): : 353 - 372
[34] Continuous time Markov decision processes with discounted moment criterion
J Math Anal Appl, 1 (01):
[35] An envelope theorem and some applications to discounted Markov decision processes
Cruz-Suarez, Hugo
Montes-de-Oca, Raul
MATHEMATICAL METHODS OF OPERATIONS RESEARCH, 2008, 67 (02) : 299 - 321
[36] Policy Gradient for Continuing Tasks in Discounted Markov Decision Processes
Paternain, Santiago
Bazerque, Juan Andres
Ribeiro, Alejandro
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2022, 67 (09) : 4467 - 4482
[37] Constrained discounted Markov decision processes with Borel state spaces
Feinberg, Eugene A.
Jaskiewicz, Anna
Nowak, Andrzej S.
AUTOMATICA, 2020, 111
[38] Countable state Markov decision processes with unbounded jump rates and discounted cost: Optimality equation and approximations
Mathematisch Instituut, Leiden University, Postbus 9512, Leiden
2300 RA, Netherlands
Adv Appl Probab, 4 (1088-1107):
[39] COUNTABLE STATE MARKOV DECISION PROCESSES WITH UNBOUNDED JUMP RATES AND DISCOUNTED COST: OPTIMALITY EQUATION AND APPROXIMATIONS
Blok, H.
Spieksma, F. M.
ADVANCES IN APPLIED PROBABILITY, 2015, 47 (04) : 1088 - 1107
[40] Simplex Algorithm for Countable-State Discounted Markov Decision Processes
Lee, Ilbin
Epelman, Marina A.
Romeijn, H. Edwin
Smith, Robert L.
OPERATIONS RESEARCH, 2017, 65 (04) : 1029 - 1042

← 1 2 3 4 5 →