A Car-following Control Algorithm Based on Deep Reinforcement Learning

Longitudinal acceleration decisions in a car-following control mode are directly determined by the state of the preceding vehicle. A driver's uncertainty makes car-following control difficult because of the complexity in state prediction of the target vehicle. To address the problem in which the performance of adaptive cruise control may deteriorate without consideration of the uncertainty of the preceding vehicle, a car-following control strategy based on deep reinforcement learning was proposed. To study the characteristics of human drivers, a driving-data-acquisition platform was established, and substantial amounts of human-driving data were collected. Based on the assumption that longitudinal control decisions are mainly affected by the preceding vehicle, a two-predecessor following structure was established. The vehicles in the driving dataset were taken as target vehicles 1# and 2# of the car-following control. Based on the real-world driving dataset, a stochastic process model was established to describe the characteristics of preceding vehicle 1# based on Gaussian process algorithm. Then car-following control was established as a Markov decision process. A car-following control method based on deep reinforcement learning was obtained through iterative learning with the stochastic process model using proximal policy optimization. Finally, the algorithm was verified based on the driving dataset. The results demonstrate that the mapping between longitudinal acceleration decisions and the states of the host and preceding vehicles can be obtained through iterative learning with consideration of the uncertainty of the target vehicle. © 2019, Editorial Department of China Journal of Highway and Transport. All right reserved.