Forced convection heat transfer control for cylinder via closed-loop continuous goal-oriented reinforcement learning

被引:1
|
作者
Liu, Yangwei [1 ,2 ]
Wang, Feitong [1 ,2 ]
Zhao, Shihang [1 ,2 ]
Tang, Yumeng [1 ,2 ]
机构
[1] Beihang Univ, Sch Energy & Power Engn, Beijing 100191, Peoples R China
[2] Beihang Univ, Natl Key Lab Sci & Technol Aeroengine Aerothermody, Beijing 100191, Peoples R China
基金
中国国家自然科学基金;
关键词
FLOW; PERFORMANCE;
D O I
10.1063/5.0239718
中图分类号
O3 [力学];
学科分类号
08 ; 0801 ;
摘要
Forced convection heat transfer control offers considerable engineering value. This study focuses on a two-dimensional rapid temperature control problem in a heat exchange system, where a cylindrical heat source is immersed in a narrow cavity. First, a closed-loop continuous deep reinforcement learning (DRL) framework based on the deep deterministic policy gradient (DDPG) algorithm is developed. This framework swiftly achieves the target temperature with a temperature variance of 0.0116, which is only 5.7% of discrete frameworks. Particle tracking technology is used to analyze the evolution of flow and heat transfer under different control strategies. Due to the broader action space for exploration, continuous algorithms inherently excel in addressing delicate control issues. Furthermore, to address the deficiency that traditional DRL-based active flow control (AFC) frameworks require retraining with each goal changes and cost substantial computational resources to develop strategies for varied goals, the goal information is directly embedded into the agent, and the hindsight experience replay (HER) is employed to improve the training stability and sample efficiency. Then, a closed-loop continuous goal-oriented reinforcement learning (GoRL) framework based on the HER-DDPG algorithm is first proposed to perform real-time rapid temperature transition control and address multiple goals without retraining. Generalization tests show the proposed GoRL framework accomplishes multi-goal tasks with a temperature variance of 0.0121, which is only 5.8% of discrete frameworks, and consumes merely 11% of the computational resources compared with frameworks without goal-oriented capability. The GoRL framework greatly enhances the ability of AFC systems to handle multiple targets and time-varying goals.
引用
收藏
页数:17
相关论文
共 50 条
  • [21] Deep reinforcement learning for closed-loop blood glucose control: two approaches
    Di Felice, Francesco
    Borri, Alessandro
    Di Benedetto, Maria Domenica
    IFAC PAPERSONLINE, 2022, 55 (40): : 115 - 120
  • [22] Probe optimization for quantum metrology via closed-loop learning control
    Xiaodong Yang
    Jayne Thompson
    Ze Wu
    Mile Gu
    Xinhua Peng
    Jiangfeng Du
    npj Quantum Information, 6
  • [23] Probe optimization for quantum metrology via closed-loop learning control
    Yang, Xiaodong
    Thompson, Jayne
    Wu, Ze
    Gu, Mile
    Peng, Xinhua
    Du, Jiangfeng
    NPJ QUANTUM INFORMATION, 2020, 6 (01)
  • [24] Optimizing passengers' experience: A goal-oriented reinforcement learning speed control approach for urban railway trains
    Liu, Wangyang
    Feng, Qingsheng
    Li, Hong
    PROCEEDINGS OF THE INSTITUTION OF MECHANICAL ENGINEERS PART F-JOURNAL OF RAIL AND RAPID TRANSIT, 2024, 238 (10) : 1283 - 1295
  • [25] Long-Term Adaptation of Closed-Loop Glucose Regulation Via Reinforcement Learning Tools
    Serafini, Maria Cecilia
    Rosales, Nicolas
    Garelli, Fabricio
    IFAC PAPERSONLINE, 2022, 55 (07): : 649 - 654
  • [26] Seismic inversion via closed-loop fully convolutional residual network and transfer learning
    Wang, Lingling
    Meng, Delin
    Wu, Bangyu
    GEOPHYSICS, 2021, 86 (05) : R671 - R683
  • [27] Convection - heat transfer coupling mechanism for closed-loop heat extraction from hydrothermal resources using horizontal wells
    Gu, Feng
    Li, Youwu
    Zhang, Yue
    Gao, Ying
    Yang, Peng
    Wang, Anran
    Cui, Jingyun
    Meitiandizhi Yu Kantan/Coal Geology and Exploration, 2024, 52 (09): : 121 - 130
  • [28] An organic brain-inspired platform with neurotransmitter closed-loop control, actuation and reinforcement learning
    Bruno, Ugo
    Rana, Daniela
    Ausilio, Chiara
    Mariano, Anna
    Bettucci, Ottavia
    Musall, Simon
    Lubrano, Claudia
    Santoro, Francesca
    MATERIALS HORIZONS, 2024, 11 (12) : 2865 - 2874
  • [29] Reinforcement learning based closed-loop reference model adaptive flight control system design
    Yuksek, Burak
    Inalhan, Gokhan
    INTERNATIONAL JOURNAL OF ADAPTIVE CONTROL AND SIGNAL PROCESSING, 2021, 35 (03) : 420 - 440
  • [30] Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators
    Thuruthel, Thomas George
    Falotico, Egidio
    Renda, Federico
    Laschi, Cecilia
    IEEE TRANSACTIONS ON ROBOTICS, 2019, 35 (01) : 124 - 134