As autonomous vehicles (AVs) are gradually merging onto the highway of reality, the problem of traffic is inching slowly (ever so slowly) on to extinction. Unfortunately, AVs will not hit the road in full force all at once. In a hundred years, every car on the road will drive itself. But between now and then, we’ll see a mixture of autonomous, semi-autonomous, and non-autonomous vehicles forced to share the road, a situation called mixed autonomy traffic.
Transportation researchers at UC Berkeley are tackling this specific situation. The researchers developed a tool called Flow, a machine learning system to help manage mixed autonomy traffic.
Herding the Human Drivers
Flow uses a type of machine learning called deep reinforcement learning, which allows the system to converge on the correct course of action as it iterates through numerous problems. In the case of Flow, the types of problems were common traffic scenarios: bottlenecks, gridlock, on-ramp merges. The agent doing the learning was a single autonomous vehicle incorporated into these traffic scenarios, and it was given input data from nearby smart vehicles and infrastructure.
Take the example of a ring road populated with regular cars. If the cars start out equidistant from one another, with all of them moving at a uniform speed, the system should, in theory, remain smooth and bottleneck free. But of course, if the cars are all driven by primates, things turn out differently. Eventually, the cars get jammed and traffic slows to a crawl.
But if you add just one autonomous vehicle (and a little bit of external data) and train it with Flow’s deep reinforcement learning, the situation changes. The AV can adjust its speed and position in just such a way so as to alleviate traffic for all the cars on the road. It’s a surprising and incredible trick, as if the AV is a shepherd herding the flock of human drivers out of traffic. This short clip offers a visual demonstration:
Managing Traffic at a Citywide Scale
Of course, toy examples like the ring road don’t do much to solve real-world problems. But Flow isn’t restricted to toy examples. The researchers also created a model of the San Francisco-Oakland Bridge, a notoriously bottlenecked thoroughfare that sees fifteen lanes merge down to five. Adding in the Flow AV helped maximize the amount of cars that could flow off the bridge.
Ultimately, the researchers hope to see Flow provide a system for managing traffic at a citywide scale. This entails simulating ever more complex scenarios. Outside collaborators can also get in on the action, taking the researcher’s benchmarks and working to find even better solutions.
“We need a more sophisticated traffic-management system, and we also need to think about how we can use these autonomous vehicles as part of traffic control,” said Eugene Vinitsky, co-lead author on the Flow study. “There are huge benefits that even 4 or 5 percent of vehicles can provide to the rest of the traffic on the road.”