A Distributed Actor-Critic Learning Approach for Affine Formation Control of Multi-Robots With Unknown Dynamics

被引:0
|
作者
Zhang, Ronghua [1 ,2 ]
Ma, Qingwen [1 ]
Zhang, Xinglong [1 ]
Xu, Xin [1 ]
Liu, Daxue [1 ]
机构
[1] Natl Univ Def Technol, Coll Intelligence Sci & Technol, Changsha, Peoples R China
[2] Sichuan Univ Sci & Engn, Sch Mech Engn, Zigong, Peoples R China
关键词
affine formation control; data-driven; multi-robots; reinforcement learning; rollout; TIME NONLINEAR-SYSTEMS;
D O I
10.1002/acs.3972
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Formation maneuverability is particularly important for multi-robots (MRs), especially when the robots are operating cooperatively in complex and dynamic environments. Although various methods have been developed for affine formation, it is still a difficult problem to design an affine formation controller for MRs with unknown dynamics. In this paper, a distributed actor-critic learning approach (DACL) in a look-ahead rollout manner is proposed for the affine formation of MRs under local communication, which improves the online learning efficiency. In the proposed approach, a distributed data-driven online optimization mechanism is designed via the sparse kernel technique to solve the near-optimal affine formation control issue of MRs with unknown dynamics as well as improve control performance. The unknown dynamics of MRs are learned offline based on precollected input-output datasets, and the sparse kernel-based approach is employed to increase the feature representation capability of the samples. Then, the proposed distributed online actor-critic algorithm for each robot in the formation includes two neural networks, which are utilized to approximate the costate functions and the near-optimal policies. Moreover, the convergence analysis of the proposed approach has been conducted. Finally, numerical simulation and KKSwarm-based experiment studies are performed to verify the effectiveness of the proposed approach.
引用
收藏
页数:15
相关论文
共 50 条
  • [31] Multi-Agent Natural Actor-Critic Reinforcement Learning Algorithms
    Prashant Trivedi
    Nandyala Hemachandra
    Dynamic Games and Applications, 2023, 13 : 25 - 55
  • [32] Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning
    Christianos, Filippos
    Schafer, Lukas
    Albrecht, Stefano V.
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020, 2020, 33
  • [33] An Object Oriented Approach to Fuzzy Actor-Critic Learning for Multi-Agent Differential Games
    Schwartz, Howard
    2019 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (IEEE SSCI 2019), 2019, : 183 - 190
  • [34] Multi-Agent Natural Actor-Critic Reinforcement Learning Algorithms
    Trivedi, Prashant
    Hemachandra, Nandyala
    DYNAMIC GAMES AND APPLICATIONS, 2023, 13 (01) : 25 - 55
  • [35] Distributed Actor-Critic Algorithms for Multiagent Reinforcement Learning Over Directed Graphs
    Dai, Pengcheng
    Yu, Wenwu
    Wang, He
    Baldi, Simone
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2023, 34 (10) : 7210 - 7221
  • [36] A multi-agent reinforcement learning using Actor-Critic methods
    Li, Chun-Gui
    Wang, Meng
    Yuan, Qing-Neng
    PROCEEDINGS OF 2008 INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND CYBERNETICS, VOLS 1-7, 2008, : 878 - 882
  • [37] An Online Actor-Critic Learning Approach with Levenberg-Marquardt Algorithm
    Ni, Zhen
    He, Haibo
    Prokhorov, Danil V.
    Fu, Jian
    2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2011, : 2333 - 2340
  • [38] Learning the Cooperative Behaviors of Seesaw Balancing Agents - An Actor-Critic Approach
    Kawakami, Takashi
    Kinoshita, Masahiro
    Kakazu, Yukinori
    INTELLIGENT AUTONOMOUS SYSTEMS 9, 2006, : 255 - +
  • [39] An Actor-Critic approach for control of Residential Photovoltaic-Battery Systems
    Joshi, Amit
    Tipaldi, Massimo
    Glielmo, Luigi
    IFAC PAPERSONLINE, 2021, 54 (07): : 222 - 227
  • [40] Distributed off-Policy Actor-Critic Reinforcement Learning with Policy Consensus
    Zhang, Yan
    Zavlanos, Michael M.
    2019 IEEE 58TH CONFERENCE ON DECISION AND CONTROL (CDC), 2019, : 4674 - 4679