EGLight: enhancing deep reinforcement learning with expert guidance for traffic signal control

Deep Reinforcement Learning (DRL) is prevalent in traffic signal control. However, the training process often encounters slow learning rate and unstable convergence due to limited state representation and exploratory learning. Inspired by human learning, we incorporate expert guidance in the exploration process to accelerate convergence and enhance performance. The proposed framework, termed Expert-Guided Light (EGLight), contains three moudles. The state perception module combines statistical features with cellular features to enhance model robustness. The decision-making module employs expert-guided learning to promote the learning efficiency. In the learning module, four distinct loss functions are employed to make full use of the interaction experience and update the agent's policy. Extensive tests demonstrate EGLight's superior convergence speed and effectiveness over traditional methods. The analysis shows that the precise feature design is helpful for the agent and proper expert guidance is crucial for the convergence of agent learning process.