Optimization of Shunting Operation Plan for Detaching and Attaching Trains Based on Q-Learning Algorithm

For the formulation of shunting operation plan of detaching and attaching trains marshalling, an optimization method for shunting operation plan was proposed based on the reinforcement learning technology and Q-learning algorithm. The shunting operation plan was divided into fall and restructure based on the tabulation method. The reinforcement learning model of the shunting operation problem was constructed by three elements including action, state, and reward. Taking the shunting locomotive as the agent, the track number of train group fall-down as the action, and the fall-down condition of the train group waiting to be marshaled as the state, the specific conditions of detaching and attaching trains and the restructuring process of the train group were established. The reward function was designed according to the connecting state of the train group fall-down and the total shunting distance after restructuring the trains. The Q-learning algorithm was improved to solve the model. The mapping relationship of the trains waiting to be marshaled and the optimal shunting operation plan was established with the goal of minimizing the shunting distance. The optimal shunting operation plan could be obtained with the adequate learning of agents. The proposed method was verified by comparing three sets of examples. The results show that compared with the overall planning and coordinating method and the binary search tree algorithm, the proposed method can get a better shunting operation plan with fewer tracks. Compared with the branch-and-bound algorithm, this method can solve the shunting operation plan with approximate quality in a shorter time. Therefore, the proposed method serves to improve the intelligent decision-making level of the shunting operation planning for stations. © 2022, Editorial Department of China Railway Science. All right reserved.