Near-Optimal Policy Optimization for Correlated Equilibrium in General-Sum Markov Games

被引：0
|
作者：
Cai, Yang [1 ]
Luo, Haipeng [2 ]
Wei, Chen-Yu [3 ]
Zheng, Weiqiang [1 ]
机构：
[1] Yale Univ, New Haven, CT 06520 USA
[2] Univ Southern Calif, Los Angeles, CA 90007 USA
[3] Univ Virginia, Charlottesville, VA USA
来源：
INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 238 | 2024年 / 238卷
关键词：
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
We study policy optimization algorithms for computing correlated equilibria in multiplayer general-sum Markov Games. Previous results achieve (O) over tilde (T-1/2) convergence rate to a correlated equilibrium and an accelerated (O) over tilde (T-3/4) convergence rate to the weaker notion of coarse correlated equilibrium. In this paper, we improve both results significantly by providing an uncoupled policy optimization algorithm that attains a near-optimal (O) over tilde (T-1) convergence rate for computing a correlated equilibrium. Our algorithm is constructed by combining two main elements (i) smooth value updates and (ii) the optimisticfollow-the-regularized-leader algorithm with the log barrier regularizer.
引用
收藏
页数：20
相关论文
共 50 条
  • [1] Near-Optimal No-Regret Learning for Correlated Equilibria in Multi-player General-Sum Games
    Anagnostides, Ioannis
    Daskalakis, Constantinos
    Farina, Gabriele
    Fishelson, Maxwell
    Golowich, Noah
    Sandholm, Tuomas
    PROCEEDINGS OF THE 54TH ANNUAL ACM SIGACT SYMPOSIUM ON THEORY OF COMPUTING (STOC '22), 2022, : 736 - 749
  • [2] On the complexity of computing Markov perfect equilibrium in general-sum stochastic games
    Deng, Xiaotie
    Li, Ningyuan
    Mguni, David
    Wang, Jun
    Yang, Yaodong
    NATIONAL SCIENCE REVIEW, 2023, 10 (01)
  • [3] On the complexity of computing Markov perfect equilibrium in general-sum stochastic games
    Xiaotie Deng
    Ningyuan Li
    David Mguni
    Jun Wang
    Yaodong Yang
    National Science Review, 2023, 10 (01) : 288 - 301
  • [4] Multiagent Reinforcement Learning for Nash Equilibrium Seeking in General-Sum Markov Games
    Moghaddam, Alireza Ramezani
    Kebriaei, Hamed
    IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS, 2025, 55 (01): : 221 - 227
  • [5] Learning Nash Equilibrium for General-Sum Markov Games from Batch Data
    Perolat, Julien
    Strub, Florian
    Piot, Bilal
    Pietquin, Olivier
    ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 54, 2017, 54 : 232 - 241
  • [6] Robustness of Markov perfect equilibrium to model approximations in general-sum dynamic games
    Subramanian, Jayakumar
    Sinha, Amit
    Mahajan, Aditya
    2021 SEVENTH INDIAN CONTROL CONFERENCE (ICC), 2021, : 189 - 194
  • [7] PAC Reinforcement Learning Algorithm for General-Sum Markov Games
    Zehfroosh, Ashkan
    Tanner, Herbert G.
    IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2023, 68 (05) : 2821 - 2831
  • [8] Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov Games
    Mao, Weichao
    Basar, Tamer
    DYNAMIC GAMES AND APPLICATIONS, 2023, 13 (01) : 165 - 186
  • [9] Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov Games
    Weichao Mao
    Tamer Başar
    Dynamic Games and Applications, 2023, 13 : 165 - 186
  • [10] Learning Stationary Correlated Equilibria in Constrained General-Sum Stochastic Games
    Hakami, Vesal
    Dehghan, Mehdi
    IEEE TRANSACTIONS ON CYBERNETICS, 2016, 46 (07) : 1640 - 1654
12345