Regret Bounds for Risk-Sensitive Reinforcement Learning

被引：0
|
作者：
Bastani, Osbert [1 ]
Ma, Yecheng Jason [1 ]
Shen, Estelle [1 ]
Xu, Wanqiao [2 ]
机构：
[1] Univ Penn, Philadelphia, PA 19104 USA
[2] Stanford Univ, Stanford, CA USA
来源：
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35, NEURIPS 2022 | 2022年
关键词：
VALUE-AT-RISK;
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
In safety-critical applications of reinforcement learning such as healthcare and robotics, it is often desirable to optimize risk-sensitive objectives that account for tail outcomes rather than expected reward. We prove the first regret bounds for reinforcement learning under a general class of risk-sensitive objectives including the popular CVaR objective. Our theory is based on a novel characterization of the CVaR objective as well as a novel optimistic MDP construction.
引用
收藏
页数：11
相关论文
共 50 条
  • [11] Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret
    Fei, Yingjie
    Yang, Zhuoran
    Chen, Yudong
    Wang, Zhaoran
    Xie, Qiaomin
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020, 2020, 33
  • [12] Inverse Risk-Sensitive Reinforcement Learning
    Ratliff, Lillian J.
    Mazumdar, Eric
    IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2020, 65 (03) : 1256 - 1263
  • [13] Risk-Sensitive Policy with Distributional Reinforcement Learning
    Theate, Thibaut
    Ernst, Damien
    ALGORITHMS, 2023, 16 (07)
  • [14] Risk-sensitive Reinforcement Learning and Robust Learning for Control
    Noorani, Erfaun
    Baras, John S.
    2021 60TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC), 2021, : 2976 - 2981
  • [15] A Probabilistic Perspective on Risk-sensitive Reinforcement Learning
    Noorani, Erfaun
    Baras, John S.
    2022 AMERICAN CONTROL CONFERENCE, ACC, 2022, : 2697 - 2702
  • [16] Distributional Reinforcement Learning for Risk-Sensitive Policies
    Lim, Shiau Hong
    Malik, Ilyas
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,
  • [17] Risk-sensitive Distributional Reinforcement Learning for Flight Control
    Seres, Peter
    Liu, Cheng
    van Kampen, Erik-Jan
    IFAC PAPERSONLINE, 2023, 56 (02): : 2013 - 2018
  • [18] Risk-Sensitive Inhibitory Control for Safe Reinforcement Learning
    Lederer, Armin
    Noorani, Erfaun
    Baras, John S.
    Hirche, Sandra
    2023 62ND IEEE CONFERENCE ON DECISION AND CONTROL, CDC, 2023, : 1040 - 1045
  • [19] Distributional Model Equivalence for Risk-Sensitive Reinforcement Learning
    Kastner, Tyler
    Erdogdu, Murat A.
    Farahmand, Amir-massoud
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [20] Minimax Regret Bounds for Reinforcement Learning
    Azar, Mohammad Gheshlaghi
    Osband, Ian
    Munos, Remi
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 70, 2017, 70
12345