Improving Traffic Signal Control With Joint-Action Reinforcement Learning

An increasing demand for mobility in urban areas leads to several issues such as traffic congestion. While many approaches exist to tackle this problem, a popular one is the adaptive control of traffic signals. This paper focuses on reinforcement learning based approaches for adaptive control. These use learning agents that act individually to adapt the signal timings to the current demand of vehicles crossing an intersection. However, in a network of intersections (thus constituting a multiagent reinforcement learning problem), as the actions of each agent have impact on the states, actions, and rewards of others, it is clear that there is a relevant non-local issue that is rarely dealt with. Thus, this paper proposes a joint learning approach in which agents learn in groups, besides also learning an individual policy. This means that we have two parallel learning processes. The group policy is implemented from time to time, thus helping agents to implicitly consider other agents' policies. A distinguishing feature of our approach is that it is not based on fixed membership in such groups; rather, groups are formed and dissolved on demand, based on their performance. Results of applying this approach are compared to a baseline and to when agents learn individually only, showing that the joint learning leads to lesser waiting time.