Machine Learning Techniques Aim to Reduce Traffic

Sick of sitting in traffic? Deep reinforcement learning may be the answer. (Image courtesy of Wikipedia.)

It's a problem we can all relate to: sitting in traffic and waiting for a green light. While waiting, you may have even pondered how you would try to improve traffic efficiency—surely there's got to be some way for everyone to get to work on time.

But ponder no longer, because a team of engineers from Tsinghua University in China has handed the problem over to machines. The team’s recent study makes use of deep reinforcement learning algorithms to optimize traffic signaling, and its promising results suggest there may be a way to arrive on time after all.

Deep Reinforcement Learning

Let's be clear: traffic is a complex problem to solve, and traffic control engineers have long worked on improving efficiency. The difficulty arises because there are two distinct and challenging tasks involved—the first step is to create a useful model of traffic flow, and the next is to somehow find a way to optimize it.

The team modeled traffic flow using a simplified simulation of an eight-lane intersection, with only red and green lights (no yellows) and vehicles only allowed to go straight through (no right, left, or U-turns were permitted).

A depiction of the eight-lane intersection simulation used in the study. Four lanes run north-south and four lanes run east-west. (Image courtesy of Tsinghua University.)

Using this simplified scenario, the team implemented reinforcement learning algorithms in order to determine signaling actions that were most beneficial to the system. This was evaluated by measuring the queuing length of traffic in both directions. By simulating different signaling situations, the algorithm aimed to minimize the length of traffic queues and therefore decrease driver wait time.

But reinforcement learning is insufficient on its own, due to the enormous complexity of realistic traffic conditions. That's why the team combined the reinforcement learning algorithms with another set, called deep learning algorithms. Deep learning makes use of neural networks to find hidden patterns in datasets. Combining the two types of algorithms is called deep reinforcement learning, and it greatly reduces the computation time needed to find optimized solutions.

Getting Rid of Rush Hour

A comparison of deep reinforcement learning algorithms (blue) and conventional reinforcement learning algorithms (red). Traffic delay time is noticeably lower for deep reinforcement learning at most, but not all, times of day. (Image courtesy of Tsinghua University.)

The team's deep reinforcement learning algorithms show real promise, outperforming conventional reinforcement learning algorithms by a fair amount. Over a full day's simulation, over 1,000 fewer vehicles came to a full stop with deep reinforcement learning, and vehicles spent an average of 13 seconds less in traffic during peak hours.

So while the dreaded rush hour may not disappear overnight, the team's efforts may go a long way towards simplifying real-world traffic flows for researchers. Using deep reinforcement learning algorithms may be just the tool traffic engineers need to make your morning commute a little more tolerable.

Meanwhile, in a very different approach to solving traffic congestion, Airbus is working on an autonomous flying car.