An Efficient Self-Evolution Method of Autonomous Driving for Any Given Algorithm

Autonomous vehicles are expected to achieve self-evolution in the real-world environment to gradually cover more complex and changing scenarios. Reinforcement learning focuses on how agents act in the environment to maximize the cumulative reward, with a great potential to achieve self-evolution ability. However, most of reinforcement learning algorithms suffer from a low sample efficiency, which greatly limits their application in autonomous driving. This paper presents an efficient self-evolution method for any given algorithm based on the combination of Soft Actor Critic (SAC) and Behavioral Cloning(BC). First, the states of the sample trajectory in the replay buffer are separated and input into the given algorithm (algorithm with fundamental performance) to get the output label of actions such that the SAC algorithm can be guided using BC to achieve fast iteration in the direction of optimization with existing basic performance. Then, the value iteration algorithm is combined to achieve the proportion allocation of mixed gradient feedback, in order to trade off exploitation and exploration. In addition, the proposed methodology is evaluated in simulation environment taking automated speed control as an example. Experiment results show that compared with SAC algorithm, the proposed method can realize more than three times of convergence efficiency improvement, while without destroying the exploration enhancement advantage of reinforcement learning algorithm, that is, the performance is improved by 20% compared with the given algorithm (Intelligent Driver Model, IDM). The proposed method can easily extended to improve any given model no matter it is model-based or learning-based algorithm.