Model-Free Guidance Method for Drones in Complex Environments Using Direct Policy Exploration and Optimization

被引:3
|
作者
Liu, Hongxun [1 ]
Suzuki, Satoshi [1 ]
机构
[1] Chiba Univ, Sch Sci & Engn, 1-33 Yayoi Cho,Inage Ku, Chiba 2638522, Japan
关键词
drones; reinforcement learning; policy optimization; model-free; traverse complex environments;
D O I
10.3390/drones7080514
中图分类号
TP7 [遥感技术];
学科分类号
081102 ; 0816 ; 081602 ; 083002 ; 1404 ;
摘要
In the past few decades, drones have become lighter, with longer hang times, and exhibit more agile performance. To maximize their capabilities during flights in complex environments, researchers have proposed various model-based perception, planning, and control methods aimed at decomposing the problem into modules and collaboratively accomplishing the task in a sequential manner. However, in practical environments, it is extremely difficult to model both the drones and their environments, with very few existing model-based methods. In this study, we propose a novel model-free reinforcement-learning-based method that can learn the optimal planning and control policy from experienced flight data. During the training phase, the policy considers the complete state of the drones and environmental information as inputs. It then self-optimizes based on a predefined reward function. In practical implementations, the policy takes inputs from onboard and external sensors and outputs optimal control commands to low-level velocity controllers in an end-to-end manner. By capitalizing on this property, the planning and control policy can be improved without the need for an accurate system model and can drive drones to traverse complex environments at high speeds. The policy was trained and tested in a simulator, as well as in real-world flight experiments, demonstrating its practical applicability. The results show that this model-free method can learn to fly effectively and that it holds great potential to handle different tasks and environments.
引用
收藏
页数:19
相关论文
共 50 条
  • [41] Direct adaptive model-free control of a class of uncertain nonlinear systems using Legendre polynomials
    Zarei, Reza
    Khorashadizadeh, Saeed
    TRANSACTIONS OF THE INSTITUTE OF MEASUREMENT AND CONTROL, 2019, 41 (11) : 3081 - 3091
  • [42] Demand Side Management using Model-Free Fuzzy Controller in a Direct Load Control Program
    Yazdkhasti, Pegah
    Diduch, Chris P.
    2020 IEEE ELECTRIC POWER AND ENERGY CONFERENCE (EPEC), 2020,
  • [43] Non-myopic Bayesian optimization using model-free reinforcement learning and its application to optimization in electrochemistry
    Cheon, Mujin
    Byun, Haeun
    Lee, Jay H.
    COMPUTERS & CHEMICAL ENGINEERING, 2024, 184
  • [44] Unit sizing of a stand-alone hybrid power system using model-free optimization
    Hakimi, S. M.
    Tafreshi, S. M. M.
    Rajati, M. R.
    GRC: 2007 IEEE INTERNATIONAL CONFERENCE ON GRANULAR COMPUTING, PROCEEDINGS, 2007, : 751 - 756
  • [45] Isothermal Kinetics of the Pentlandite Exsolution from mss/Pyrrhotite Using Model-Free Method
    WANG Haipeng School of Chemical Engineering
    Tsinghua Science and Technology, 2006, (03) : 368 - 373
  • [46] Isothermal kinetics of the pentlandite exsolution from mss/pyrrhotite using model-free method
    School of Chemical Engineering, University of Adelaide, Adelaide, SA 5005, Australia
    Tsinghua Sci. Tech., 2006, 3 (368-373):
  • [47] A model-free sampling method for basins of attraction using hybrid active learning (HAL)
    Wang, Xue-She
    Moore, Samuel A.
    Turner, James D.
    Mann, Brian P.
    COMMUNICATIONS IN NONLINEAR SCIENCE AND NUMERICAL SIMULATION, 2022, 112
  • [48] Model-free optimization of power/efficiency tradeoffs in quantum thermal machines using reinforcement learning
    Erdman, Paolo A.
    Noe, Frank
    PNAS NEXUS, 2023, 2 (08):
  • [49] Model-Free Optimization Scheme for Efficiency Improvement of Wind Farm Using Decentralized Reinforcement Learning
    Xu, Zhiwei
    Geng, Hua
    Chu, Bing
    Qian, Menghao
    Tan, Ni
    IFAC PAPERSONLINE, 2020, 53 (02): : 12103 - 12108
  • [50] A decentralized, model-free, global optimization method for energy saving in heating, ventilation and air conditioning systems
    Wang, Shiqiang
    Xing, Jianchun
    Jiang, Ziyan
    Dai, Yunchuang
    BUILDING SERVICES ENGINEERING RESEARCH & TECHNOLOGY, 2020, 41 (04): : 414 - 428