Optimal tracking agent: a new framework of reinforcement learning for multiagent systems

被引:3
|
作者
Cao, Weihua [1 ]
Chen, Gang [1 ]
Chen, Xin [1 ]
Wu, Min [1 ]
机构
[1] Cent South Univ, Inst Adv Control & Intelligent Automat, Sch Informat Sci & Engn, Changsha 410083, Hunan, Peoples R China
来源
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE | 2013年 / 25卷 / 14期
基金
高等学校博士学科点专项科研基金;
关键词
estimator; action selection mechanism; curse of dimensionality; optimal tracking agent; multiagent systems;
D O I
10.1002/cpe.2870
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
SUMMARYThe curse of dimensionality is a ubiquitous problem for multiagent reinforcement learning, which means the learning and storing space grows exponentially with the number of agents and hinders the application of multiagent reinforcement learning. To relieve this problem, we propose a new framework named as optimal tracking agent (OTA). The OTA views the other agents as part of the environment and uses a reduced form to learn the optimal decision. Although merging other agents into the environment may reduce the dimension of action space, the environment characterized by such form is dynamic and does not satisfy the convergence of reinforcement learning (RL). Thus, we develop an estimator to track the dynamics of the environment. The estimator obtains the dynamic model, and then the model-based RL can be used to react to the dynamic environment optimally. Because the Q-function in OTA is also a dynamic process because of other agents' dynamics, different from traditional RL, in which the learning is a stationary process and the usual action selection mechanisms just suit to such stationary process, we improve the greedy action selection mechanism to adapt to such dynamics. Thus, the OTA will have convergence. An experiment illustrates the validity and efficiency of the OTA.Copyright (c) 2012 John Wiley & Sons, Ltd.
引用
收藏
页码:2002 / 2015
页数:14
相关论文
共 50 条
  • [31] Multiagent Reinforcement Social Learning toward Coordination in Cooperative Multiagent Systems
    Hao, Jianye
    Leung, Ho-Fung
    Ming, Zhong
    ACM TRANSACTIONS ON AUTONOMOUS AND ADAPTIVE SYSTEMS, 2015, 9 (04)
  • [32] Adaptive Multigradient Recursive Reinforcement Learning Event-Triggered Tracking Control for Multiagent Systems
    Li, Hongyi
    Wu, Ying
    Chen, Mou
    Lu, Renquan
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2023, 34 (01) : 144 - 156
  • [33] PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems
    Biagioni, David
    Zhang, Xiangyu
    Wald, Dylan
    Vaidhynathan, Deepthi
    Chintala, Rohit
    King, Jennifer
    Zamzam, Ahmed S.
    PROCEEDINGS OF THE 2022 THE THIRTEENTH ACM INTERNATIONAL CONFERENCE ON FUTURE ENERGY SYSTEMS, E-ENERGY 2022, 2022, : 565 - 570
  • [34] N-learning: A reinforcement learning paradigm for multiagent systems
    Mansfield, M
    Collins, JJ
    Eaton, M
    Collins, T
    AI 2005: ADVANCES IN ARTIFICIAL INTELLIGENCE, 2005, 3809 : 684 - 694
  • [35] Implementing Traffic Signal Optimal Control by Multiagent Reinforcement Learning
    Song, Jiong
    Jin, Zhao
    Zhu, WenJun
    2011 INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND NETWORK TECHNOLOGY (ICCSNT), VOLS 1-4, 2012, : 2578 - 2582
  • [36] Data-Based Optimal Consensus Control for Multiagent Systems With Policy Gradient Reinforcement Learning
    Yang, Xindi
    Zhang, Hao
    Wang, Zhuping
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, 33 (08) : 3872 - 3883
  • [37] Adaptive Optimal Consensus Control of Multiagent Systems With Unknown Dynamics and Disturbances via Reinforcement Learning
    Chen L.
    Dong C.
    Dai S.-L.
    IEEE Transactions on Artificial Intelligence, 2024, 5 (05): : 2193 - 2203
  • [38] Prescribed-Time Optimal Consensus for Switched Stochastic Multiagent Systems: Reinforcement Learning Strategy
    Guang, Weiwei
    Wang, Xin
    Tan, Lihua
    Sun, Jian
    Huang, Tingwen
    IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE, 2025, 9 (01): : 75 - 86
  • [39] Reinforcement learning-based optimal tracking control for uncertain multi-agent systems with uncertain topological networks
    You, Renyang
    Liu, Quan
    ISA TRANSACTIONS, 2025, 156 : 217 - 227
  • [40] DTDE: A new cooperative multi-agent reinforcement learning framework
    Wen, Guanghui
    Fu, Junjie
    Dai, Pengcheng
    Zhou, Jialing
    INNOVATION, 2021, 2 (04):