Regret Bounds for Risk-Sensitive Reinforcement Learning

被引：0

作者：

Bastani, Osbert ^{[1
]}

Ma, Yecheng Jason ^{[1
]}

Shen, Estelle ^{[1
]}

Xu, Wanqiao ^{[2
]}

机构：

[1] Univ Penn, Philadelphia, PA 19104 USA

[2] Stanford Univ, Stanford, CA USA

来源：

ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35, NEURIPS 2022 | 2022年

关键词：

VALUE-AT-RISK;

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

In safety-critical applications of reinforcement learning such as healthcare and robotics, it is often desirable to optimize risk-sensitive objectives that account for tail outcomes rather than expected reward. We prove the first regret bounds for reinforcement learning under a general class of risk-sensitive objectives including the popular CVaR objective. Our theory is based on a novel characterization of the CVaR objective as well as a novel optimistic MDP construction.

引用

页数：11

共 50 条

[21] Variational Regret Bounds for Reinforcement Learning
Ortner, Ronald
Gajane, Pratik
Auer, Peter
35TH UNCERTAINTY IN ARTIFICIAL INTELLIGENCE CONFERENCE (UAI 2019), 2020, 115 : 81 - 90
[22] RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agents
Qiu, Wei
Wang, Xinrun
Yu, Runsheng
He, Xu
Wang, Rundong
An, Bo
Obraztsova, Svetlana
Rabinovich, Zinovi
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 34 (NEURIPS 2021), 2021, 34
[23] Risk-sensitive Inverse Reinforcement Learning via Coherent Risk Models
Majumdar, Anirudha
Singh, Sumeet
Mandlekar, Ajay
Pavone, Marco
ROBOTICS: SCIENCE AND SYSTEMS XIII, 2017,
[24] Risk-sensitive reinforcement learning algorithms with generalized average criterion
殷苌茗
王汉兴
赵飞
AppliedMathematicsandMechanics(EnglishEdition), 2007, (03) : 405 - 416
[25] State-Augmentation Transformations for Risk-Sensitive Reinforcement Learning
Ma, Shuai
Yu, Jia Yuan
THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FIRST INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / NINTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2019, : 4512 - 4519
[26] Risk-sensitive reinforcement learning algorithms with generalized average criterion
Chang-ming Yin
Wang Han-xing
Zhao Fei
Applied Mathematics and Mechanics, 2007, 28 : 405 - 416
[27] Gradient-Based Inverse Risk-Sensitive Reinforcement Learning
Mazumdar, Eric
Ratliff, Lillian J.
Fiez, Tanner
Sastry, S. Shankar
2017 IEEE 56TH ANNUAL CONFERENCE ON DECISION AND CONTROL (CDC), 2017,
[28] Risk-Sensitive Reinforcement Learning via Policy Gradient Search
Prashanth, L. A.
Fu, Michael C.
FOUNDATIONS AND TRENDS IN MACHINE LEARNING, 2022, 15 (05): : 537 - 693
[29] Risk-sensitive reinforcement learning algorithms with generalized average criterion
Yin Chang-ming
Wang Han-xing
Zhao Fei
APPLIED MATHEMATICS AND MECHANICS-ENGLISH EDITION, 2007, 28 (03) : 405 - 416
[30] Risk-Sensitive Reinforcement Learning with Function Approximation: A Debiasing Approach
Fei, Yingjie
Yang, Zhuoran
Wang, Zhaoran
INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139, 2021, 139

← 1 2 3 4 5 →