Hierarchical residual reinforcement learning based path tracking control method for unmanned bicycle

Due to their super under-actuation and high nonlinearity properties, simplification and linearization techniques are necessary in the traditional model-based path tracking controller design of unmanned bicycles, leading to the decrease of their control performance. On the other hand, as one of the most important learning-based methods, deep reinforcement learning (DRL) suffers from low training efficiency and initial safety issues. In this letter, a hierarchical residual reinforcement learning (HRRL)-based path tracking control method is proposed, to address the drawbacks of both traditional and learning-based approaches. The path tracking task is decomposed into two subtasks, i.e., attitude control and position control, and the controllers are designed separately for each subtask. In each controller, a DRL controller is connected to a traditional controller through residual connection. Physical simulation experiments demonstrate that compared to the traditional LQR, LQI, Stanley and DRL approaches, the proposed method can improve the tracking performance of unmanned bicycles and decrease the training time and the tipping number during training. Furthermore, experimental results also show that the proposed controller exhibits a certain level of robustness, enabling effective path tracking in complex terrain. © 2025 Elsevier B.V.