A Distributed Actor-Critic Learning Approach for Affine Formation Control of Multi-Robots With Unknown Dynamics

被引:0
|
作者
Zhang, Ronghua [1 ,2 ]
Ma, Qingwen [1 ]
Zhang, Xinglong [1 ]
Xu, Xin [1 ]
Liu, Daxue [1 ]
机构
[1] Natl Univ Def Technol, Coll Intelligence Sci & Technol, Changsha, Peoples R China
[2] Sichuan Univ Sci & Engn, Sch Mech Engn, Zigong, Peoples R China
关键词
affine formation control; data-driven; multi-robots; reinforcement learning; rollout; TIME NONLINEAR-SYSTEMS;
D O I
10.1002/acs.3972
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Formation maneuverability is particularly important for multi-robots (MRs), especially when the robots are operating cooperatively in complex and dynamic environments. Although various methods have been developed for affine formation, it is still a difficult problem to design an affine formation controller for MRs with unknown dynamics. In this paper, a distributed actor-critic learning approach (DACL) in a look-ahead rollout manner is proposed for the affine formation of MRs under local communication, which improves the online learning efficiency. In the proposed approach, a distributed data-driven online optimization mechanism is designed via the sparse kernel technique to solve the near-optimal affine formation control issue of MRs with unknown dynamics as well as improve control performance. The unknown dynamics of MRs are learned offline based on precollected input-output datasets, and the sparse kernel-based approach is employed to increase the feature representation capability of the samples. Then, the proposed distributed online actor-critic algorithm for each robot in the formation includes two neural networks, which are utilized to approximate the costate functions and the near-optimal policies. Moreover, the convergence analysis of the proposed approach has been conducted. Finally, numerical simulation and KKSwarm-based experiment studies are performed to verify the effectiveness of the proposed approach.
引用
收藏
页数:15
相关论文
共 50 条
  • [21] Model-Based Actor-Critic Learning for Optimal Tracking Control of Robots With Input Saturation
    Zhao, Xingwei
    Tao, Bo
    Qian, Lu
    Ding, Han
    IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2021, 68 (06) : 5046 - 5056
  • [22] Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning
    Xiao, Yuchen
    Tan, Weihao
    Amato, Christopher
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 35 (NEURIPS 2022), 2022,
  • [23] Distributed Structured Actor-Critic Reinforcement Learning for Universal Dialogue Management
    Chen, Zhi
    Chen, Lu
    Liu, Xiaoyuan
    Yu, Kai
    IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2020, 28 : 2400 - 2411
  • [24] A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning
    Lin, Yixuan
    Zhang, Kaiqing
    Yang, Zhuoran
    Wang, Zhaoran
    Basar, Tamer
    Sandhu, Romeil
    Liu, Ji
    2019 IEEE 58TH CONFERENCE ON DECISION AND CONTROL (CDC), 2019, : 5562 - 5567
  • [25] A Multi-Agent Off-Policy Actor-Critic Algorithm for Distributed Reinforcement Learning
    Suttle, Wesley
    Yang, Zhuoran
    Zhang, Kaiqing
    Wang, Zhaoran
    Basar, Tamer
    Liu, Ji
    IFAC PAPERSONLINE, 2020, 53 (02): : 1549 - 1554
  • [26] Fully distributed actor-critic architecture for multitask deep reinforcement learning
    Valcarcel Macua, Sergio
    Davies, Ian
    Tukiainen, Aleksi
    De Cote, Enrique Munoz
    KNOWLEDGE ENGINEERING REVIEW, 2021, 36
  • [27] A Kalman Filter-based Actor-Critic Learning Approach
    Wang, Bin
    Zhao, Dongbin
    PROCEEDINGS OF THE 2014 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2014, : 3657 - 3662
  • [28] Actor-critic reinforcement learning for the feedback control of a swinging chain
    Dengler, C.
    Lohmann, B.
    IFAC PAPERSONLINE, 2018, 51 (13): : 378 - 383
  • [29] Actor-critic learning based PID control for robotic manipulators
    Nohooji, Hamed Rahimi
    Zaraki, Abolfazl
    Voos, Holger
    APPLIED SOFT COMPUTING, 2024, 151
  • [30] Actor-Critic Algorithms for Constrained Multi-agent Reinforcement Learning
    Diddigi, Raghuram Bharadwaj
    Reddy, D. Sai Koti
    Prabuchandran, K. J.
    Bhatnagar, Shalabh
    AAMAS '19: PROCEEDINGS OF THE 18TH INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS, 2019, : 1931 - 1933