Location-Based Real-Time Updated Advising Method for Traffic Signal Control

Adaptive traffic signal control (ATSC) attempts to alleviate traffic congestion by dynamically adjusting the timing of traffic lights in real time, and multiagent reinforcement learning is one of the ways these systems learn how and when to change signals. However, traffic congestion continues to be a problem in most highly populated cities. We know that the current research into ATSC still has much ground to cover in terms of traffic efficiency, global optimality, and convergence stability. Hence, in this article, we outline a method that provides an advising method to the multiagent traffic signal control based on relative location in real time. ATSC is regarded as a multiagent environment, in which each traffic intersection is an agent to observe the distribution of the number of vehicles (state) at the intersection to control the change of signal lights (action). In our learning framework, each agent can not only take action by its advantage actor-critic model but can also ask its neighboring agent for advice when it is not confident in its decision. The advice is generated by a real-time updated advising model, which is based on the state and relative location of neighboring agents. Because the advising model provides real-time feedback, we find that learning is more effective and convergence is more stable. Moreover, drawing on neighboring states during taking action avoids falling into a local optimality caused by only observing local states. Comparisons with similar methods show that our method brings a significant improvement in a range of evaluation criteria, such as queue lengths, vehicle speeds, and trip delays.