Ship cooperative collision avoidance strategy based on multi-agent deep reinforcement learning

被引：0
|
作者：
Sui L.-R. [1 ]
Gao S. [1 ]
He W. [2 ]
机构：
[1] School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan
[2] College of Physics Electronic Information Engineering, Minjiang University, Fuzhou
来源：
Kongzhi yu Juece/Control and Decision | 2023年 / 38卷 / 05期
关键词：
collaborative collision avoidance strategy; collaborative decision-making; multi-agent communication model; multi-agent cooperation; multi-agent deep reinforcement learning; ship collision avoidance;
D O I：
10.13195/j.kzyjc.2022.1159
中图分类号：
学科分类号：
摘要：
Ship collision avoidance is the primary issue in intelligent navigation. In multi-ship encounters, only by collaborating and jointly planning collision avoidance strategies, the collision risk can be effectively reduced. In order to make the ship intelligent collision avoidance strategy collaborative, safe and practical, a ship collaborative collision avoidance decision method based on multi-agent deep reinforcement learning is proposed. Firstly, the method of identifying ship encounter situations is studied and a multi-ship collision avoidance strategy that satisfies the "International regulations for preventing collisions at sea" is designed. Secondly, by analysing the cooperation mode of multi-ship agents, a multi-ship agent cooperative collision avoidance decision-making model is constructed. The model uses the attention inference method to extract the key data that is helpful for collision avoidance decisions. And a memory driven experience learning method is designed to effectively accumulate interactive experience. In addition, the noise network and multihead attention mechanism are introduced into the model to enhance decision-making and exploration capabilities of ship agents. Finally, on the experimental map and the real nautical chart, simulation experiments are carried out on the multi-ship encounter scenarios. The results show that in terms of collaboration and safety, compared with multiple comparison methods, competitive results are obtained and the practical requirements are met using the proposed method, which provides a new solution for improving theintelligent navigation of ships and ensuring navigation safety. © 2023 Northeast University. All rights reserved.
引用
收藏
页码：1395 / 1402
页数：7
相关论文
共 18 条
  • [11] Rangwala M, Williams R., Learning multi-agent communication through structured attentive reasoning, Proceedings of the 34th International Conference on Neural Information Processing Systems, pp. 10088-10098, (2020)
  • [12] Ding Z L, Huang T J, Lu Z Q., Learning individually inferred communication for multi-agent cooperation[J/OL], (2020)
  • [13] Liu I J, Jain U, Yeh R A, Et al., Cooperative exploration for multi-agent deep reinforcement learning[J/OL], (2021)
  • [14] Pesce E, Montana G., Improving coordination in small-scale multi-agent deep reinforcement learning through memory-driven communication, Machine Learning, 109, 9, pp. 1727-1747, (2020)
  • [15] Kuba J G, Chen R Q, Wen M N, Et al., Trust region policy optimisation in multi-agent reinforcement learning[J/OL], (2021)
  • [16] Hu J, Hu S Y, Liao S W., Policy regularization via noisy advantage values for cooperative multi-agent actor-critic methods[J/OL], (2021)
  • [17] Iqbal S, Sha F., Actor-attention-critic for multi-agent reinforcement learning, (2018)
  • [18] Kuba J G, Chen R Q, Wen M N, Et al., Trust region policy optimisation in multi-agent reinforcement learning[J/OL], (2021)
12