Cooperative Model-Based Reinforcement Learning for Approximate Optimal Tracking

被引：0
|
作者：
Greene, Max L. [1 ]
Bell, Zachary, I [2 ]
Nivison, Scott A. [2 ]
How, Jonathan P. [3 ]
Dixon, Warren E. [1 ]
机构：
[1] Univ Florida, Dept Mech & Aerosp Engn, Gainesville, FL 32611 USA
[2] Air Force Res Lab, Munit Directorate, Eglin AFB, FL USA
[3] MIT, Dept Aeronaut & Astronaut, Cambridge, MA 02139 USA
来源：
2021 AMERICAN CONTROL CONFERENCE (ACC) | 2021年
关键词：
SYSTEMS;
D O I：
暂无
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
This paper provides an approximate online adaptive solution to the infinite-horizon optimal tracking problem for a set of agents with homogeneous dynamics and common tracking objectives. Model-based reinforcement learning is implemented by simultaneously evaluating the Bellman error (BE) at the state of each agent and on nearby off-trajectory points, as needed, throughout the state space. Each agent will calculate and share their respective on and off-trajectory BE information with a centralized estimator, which computes updates for the approximate solution to the infinite-horizon optimal tracking problem and shares the estimate with the agents. In doing so, the computational burden associated with BE extrapolation is shared between the agents and a centralized updating resource. Edge computing is leveraged to share the computational load between the agents and a centralized resource. Uniformly ultimately bounded tracking of each agent's state to the desired state and convergence of the control policy to the neighborhood of the optimal policy is proven via a Lyapunov-like stability analysis.
引用
收藏
页码：1973 / 1978
页数：6
相关论文
共 50 条
  • [1] Model-Based Reinforcement Learning for Infinite-Horizon Approximate Optimal Tracking
    Kamalapurkar, Rushikesh
    Andrews, Lindsey
    Walters, Patrick
    Dixon, Warren E.
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2017, 28 (03) : 753 - 758
  • [2] Model-based reinforcement learning for infinite-horizon approximate optimal tracking
    Kamalapurkar, Rushikesh
    Andrews, Lindsey
    Walters, Patrick
    Dixon, Warren E.
    2014 IEEE 53RD ANNUAL CONFERENCE ON DECISION AND CONTROL (CDC), 2014, : 5083 - 5088
  • [3] Model-based reinforcement learning for approximate optimal regulation
    Kamalapurkar, Rushikesh
    Walters, Patrick
    Dixon, Warren E.
    AUTOMATICA, 2016, 64 : 94 - 104
  • [4] Efficient model-based reinforcement learning for approximate online optimal control
    Kamalapurkar, Rushikesh
    Rosenfeld, Joel A.
    Dixon, Warren E.
    AUTOMATICA, 2016, 74 : 247 - 258
  • [5] Model-Based Reinforcement Learning for Approximate Optimal Control with Temporal Logic Specifications
    Cohen, Max H.
    Belta, Cahn
    HSCC2021: PROCEEDINGS OF THE 24TH INTERNATIONAL CONFERENCE ON HYBRID SYSTEMS: COMPUTATION AND CONTROL (PART OF CPS-IOT WEEK), 2021,
  • [6] Model-Based Reinforcement Learning with a Generative Model is Minimax Optimal
    Agarwal, Alekh
    Kakade, Sham
    Yang, Lin F.
    CONFERENCE ON LEARNING THEORY, VOL 125, 2020, 125
  • [7] Model-Based Reinforcement Learning for Trajectory Tracking of Musculoskeletal Robots
    Xu, Haoran
    Fan, Jianyin
    Wang, Qiang
    2023 IEEE INTERNATIONAL INSTRUMENTATION AND MEASUREMENT TECHNOLOGY CONFERENCE, I2MTC, 2023,
  • [8] Cooperative Multi-Agent Reinforcement Learning With Approximate Model Learning
    Park, Young Joon
    Lee, Young Jae
    Kim, Seoung Bum
    IEEE ACCESS, 2020, 8 : 125389 - 125400
  • [9] Model-Based Reinforcement Learning for Optimal Feedback Control of Switched Systems
    Greene, Max L.
    Abudia, Moad
    Kamalapurkar, Rushikesh
    Dixon, Warren E.
    2020 59TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC), 2020, : 162 - 167
  • [10] Model-Based Cross-Scale Reinforcement Learning Optimal Control
    Li, Gonghe
    Zhou, Linna
    Liu, Xiaomin
    Yang, Chunyu
    2024 6th International Conference on Electronic Engineering and Informatics, EEI 2024, 2024, : 906 - 910
12345