Continuous interval type-2 fuzzy Q-learning algorithm for trajectory tracking tasks for vehicles

Trajectory tracking is a fundamental but challenging task for vehicle automation. In addition to the system nonlinearity, the main difficulties in the trajectory tracking task are due to the environmental noise and the model uncertainties under different driving scenarios. Considering the uncertainties in the environment, the reinforcement learning method with continuous action and noise-resistance capability could be a promising way to overcome these issues. In this article, a novel continuous interval type-2 fuzzy Q-learning (CIT2FQL) algorithm is proposed to deal with the trajectory tracking task. By introducing the n-dimensional interval type-2 fuzzy inference system (n-D IT2FIS) in fuzzy Q-learning, our proposed method achieves the continuous Q-learning by combining the action interpolation with IT2FIS for the first time. We also proposed a simplified type-reduction method for n-D IT2FIS to improve the computing efficiency of the proposed method. Moreover, a radial basis function (RBF) layer is chosen as the basis function to achieve the q-value interpolation. Finally, a trajectory tracking task in a simulation environment is conducted to verify the effectiveness and robustness of the proposed method under different scenarios. The results demonstrate that the proposed method has better robustness and noise-resistance capability while maintaining good tracking performance compared with the state-of-the-art baseline algorithms including double deep Q network (DDQN), proximal policy optimization (PPO), and interval type-2 dynamic fuzzy Q-learning (IT2DFQL).