Forced convection heat transfer control for cylinder via closed-loop continuous goal-oriented reinforcement learning

被引:1
|
作者
Liu, Yangwei [1 ,2 ]
Wang, Feitong [1 ,2 ]
Zhao, Shihang [1 ,2 ]
Tang, Yumeng [1 ,2 ]
机构
[1] Beihang Univ, Sch Energy & Power Engn, Beijing 100191, Peoples R China
[2] Beihang Univ, Natl Key Lab Sci & Technol Aeroengine Aerothermody, Beijing 100191, Peoples R China
基金
中国国家自然科学基金;
关键词
FLOW; PERFORMANCE;
D O I
10.1063/5.0239718
中图分类号
O3 [力学];
学科分类号
08 ; 0801 ;
摘要
Forced convection heat transfer control offers considerable engineering value. This study focuses on a two-dimensional rapid temperature control problem in a heat exchange system, where a cylindrical heat source is immersed in a narrow cavity. First, a closed-loop continuous deep reinforcement learning (DRL) framework based on the deep deterministic policy gradient (DDPG) algorithm is developed. This framework swiftly achieves the target temperature with a temperature variance of 0.0116, which is only 5.7% of discrete frameworks. Particle tracking technology is used to analyze the evolution of flow and heat transfer under different control strategies. Due to the broader action space for exploration, continuous algorithms inherently excel in addressing delicate control issues. Furthermore, to address the deficiency that traditional DRL-based active flow control (AFC) frameworks require retraining with each goal changes and cost substantial computational resources to develop strategies for varied goals, the goal information is directly embedded into the agent, and the hindsight experience replay (HER) is employed to improve the training stability and sample efficiency. Then, a closed-loop continuous goal-oriented reinforcement learning (GoRL) framework based on the HER-DDPG algorithm is first proposed to perform real-time rapid temperature transition control and address multiple goals without retraining. Generalization tests show the proposed GoRL framework accomplishes multi-goal tasks with a temperature variance of 0.0121, which is only 5.8% of discrete frameworks, and consumes merely 11% of the computational resources compared with frameworks without goal-oriented capability. The GoRL framework greatly enhances the ability of AFC systems to handle multiple targets and time-varying goals.
引用
收藏
页数:17
相关论文
共 50 条
  • [41] Model-free closed-loop wind farm control using reinforcement learning with recursive least squares
    Liew, Jaime
    Gocmen, Tuhfe
    Lio, Wai Hou
    Larsen, Gunner Chr.
    WIND ENERGY, 2024, 27 (11) : 1173 - 1187
  • [42] Closed-loop Flow Control Method Based on Deep Reinforcement Learning using a Co-flow Jet
    Zhao, Y. R.
    Xu, H. Y.
    Xie, Z. Y.
    JOURNAL OF APPLIED FLUID MECHANICS, 2024, 17 (04) : 816 - 827
  • [43] Close d-loop force d heat convection control using deep reinforcement learning
    Wang, Yi-Zhe
    He, Xian-Jun
    Hua, Yue
    Chen, Zhi-Hua
    Wu, Wei-Tao
    Zhou, Zhi-Fu
    INTERNATIONAL JOURNAL OF HEAT AND MASS TRANSFER, 2023, 202
  • [44] Early fault detection of current sensors operating in a closed-loop control structure using transfer learning
    Skowron, Maciej
    Teler, Krystian
    Orlowska-Kowalska, Teresa
    2024 IEEE 21ST INTERNATIONAL POWER ELECTRONICS AND MOTION CONTROL CONFERENCE, PEMC 2024, 2024,
  • [45] CSGP: Closed-Loop Safe Grasp Planning via Attention-Based Deep Reinforcement Learning From Demonstrations
    Tang, Zixin
    Shi, Yifei
    Xu, Xin
    IEEE ROBOTICS AND AUTOMATION LETTERS, 2023, 8 (06) : 3158 - 3165
  • [46] Two-dimensional simulation of fluid flow and forced convection heat transfer from a circular cylinder with control rods
    Harimi, Somayeh
    Marjani, Azam
    Moradi, Sadegh
    INTERNATIONAL JOURNAL OF NUMERICAL METHODS FOR HEAT & FLUID FLOW, 2018, 28 (04) : 799 - 827
  • [47] Two-dimensional simulation of fluid flow and forced convection heat transfer from a circular cylinder with control rods
    Harimi S.
    Marjani A.
    Moradi S.
    Harimi, Somayeh (S-Harimi91@iau-arak.ac.ir), 2018, Emerald Publishing (28): : 799 - 827
  • [49] Closed-Loop Soft Robot Control Frameworks with Coordinated Policies Based on Reinforcement Learning and Proprioceptive Self-Sensing
    Ju, Hunpyo
    Cha, Baekdong
    Rus, Daniela
    Lee, Jongho
    ADVANCED FUNCTIONAL MATERIALS, 2023,
  • [50] Two-degree-of-freedom flow-induced vibration suppression of a circular cylinder via externally forced rotational oscillations: comparison of active open-loop and closed-loop control systems
    Amir Hossein Rabiee
    Journal of the Brazilian Society of Mechanical Sciences and Engineering, 2020, 42