Yard Crane Scheduling Method Based on Deep Reinforcement Learning for the Automated Container Terminal

被引：0
|
作者：
Wang W. [1 ]
Huang Z. [1 ]
Zhuang Z. [1 ]
Fang H. [2 ]
Qin W. [1 ]
机构：
[1] Institute of Industrial Engineering and Management, Shanghai Jiao Tong University, Shanghai
[2] Shanghai International Port (Group) Co., Ltd., Shanghai
来源：
Jixie Gongcheng Xuebao/Journal of Mechanical Engineering | 2024年 / 60卷 / 06期
关键词：
automated container terminal; deep reinforcement learning; yard; yard crane scheduling;
D O I：
10.3901/JME.2024.06.044
中图分类号：
学科分类号：
摘要：
As the core working machinery of automated terminal yard, the dispatching of yard crane is the key to improve the efficiency of container operation. In order to minimize the waiting time of AGVs and external container trucks, a mathematical programming model for the yard crane scheduling problem is established considering complex spatio-temporal coupling characteristics and high dynamic, and a novel deep reinforcement learning method is proposed to solve the problem. The algorithm describes the yard environment close to reality through the agent definition, and improves the ability of extracting hidden state patterns through pointer network, attention mechanism and A-C architecture in the interaction design between the agent and the environment. Experiments are carried out on examples of different scales based on the actual data of Yangshan Phase IV Automated Terminal. The results show that the proposed algorithm can provide an approximately optimal crane scheduling scheme in a relatively short time, and the performance of it is about 17% better compared with state-of-art heuristic rule algorithms. Therefore, the proposed scheduling method is effective and superior, and it can provide dynamic decision support for yard operation in practice. © 2024 Chinese Mechanical Engineering Society. All rights reserved.
引用
收藏
页码：44 / 57
页数：13
相关论文
共 26 条
  • [11] HU Z H, SHEU J B, LUO J X., Sequencing twin automated stacking cranes in a block at automated container terminal[J], Transportation Research Part C：Emerging Technologies, 69, pp. 208-227, (2016)
  • [12] LU H, WANG S., A Study on multi-ASC scheduling method of automated container terminals based on graph theory[J], Computers & Industrial Engineering, 129, pp. 404-416, (2019)
  • [13] ZHENG F，, MAN X, CHU F，, Et al., Two yard crane scheduling with dynamic processing time and interference[J], IEEE Transactions on Intelligent Transportation Systems, 19, 12, pp. 3775-3784, (2018)
  • [14] HAN X, WANG Q, HUANG J., Scheduling cooperative twin automated stacking cranes in automated container terminals[J], Computers & Industrial Engineering, 128, pp. 553-558, (2019)
  • [15] HE J, TAN C, ZHANG Y., Yard crane scheduling problem in a container terminal considering risk caused by uncertainty[J], Advanced Engineering Informatics, 39, pp. 14-24, (2019)
  • [16] ZHENG F，, MAN X, CHU F，, Et al., A two-stage stochastic programming for single yard crane scheduling with uncertain release times of retrieval tasks[J], International Journal of Production Research, 57, 13, pp. 4132-4147, (2019)
  • [17] SUTTON R S, BARTO A G., Reinforcement learning：An introduction[M], (2018)
  • [18] LIN X, BELING P A, COGILL R, Et al., Multiagent inverse reinforcement learning for two-person zero-sum games[J], IEEE Transactions on Computational Intelligence and AI in Games, 10, 1, pp. 56-68, (2017)
  • [19] NAZARI M, OROOJLOOY A, SNYDER L V, Et al., Reinforcement learning for solving the vehicle routing problem[C], Advances in Neural Information Processing Systems 31, pp. 9861-9871, (2018)
  • [20] HWANG I, JANG Y J., Q(λ) learning-based dynamic route guidance algorithm for overhead hoist transport systems in semiconductor fabs[J], International Journal of Production Research, 58, 4, pp. 1199-1221, (2020)
123