This was accomplished by incorporating deep learning algorithms into a cascade detection program.
"We're aiming to build computer vision systems that will help computers better understand the world around them," said research director Nuno Vasconcelos. “A big goal is real-time vision especially for pedestrian detection systems in self-driving cars.”
New Algorithm with Deep Learning Models
The new pedestrian detection algorithm breaks an image down into small windows for processing by a classifier, which signals whether a pedestrian is present or not. Since pedestrians appear in different sizes depending on the camera’s distance, millions of windows are inspected frame by frame at speeds between five and thirty frames per second.
Cascade detection operates over three main stages.
First, the algorithm quickly identifies windows which are clearly not pedestrians, such as the sky. Next, the program recognizes objects that have person-like features, such as trees and (obviously) pedestrians. In the last stage, windows from the second stage are analyzed to determine the difference between pedestrians and very similar objects.
The classifiers employed in this process are “weak learners” —simple and fast methods for classification. However, for the last stages with higher complexity requirements, weak learners do not provide the best results.
To solve this problem, Vasconcelos and his team developed an algorithm that uses deep learning models in the final stage. While deep learning models are better at detecting whether a person is present in a frame or not, implementing such a complex program in real time is challenging, especially in the early stages.
However, by combining the simple classifiers in the early stages with the complex classifiers in the last stage, the researchers were able to create a new cascade algorithm for optimal detection accuracy and complexity.
"No previous algorithms have been capable of optimizing the trade-off between detection accuracy and speed for cascades with stages of such different complexities,” said Vasconcelos. “In fact, these are the first cascades to include stages of deep learning. The results obtained with this new algorithm are substantially better for real-time, accurate pedestrian detection."
Pedestrian Detection in Autonomous Vehicles
The researchers are now working to further develop their technology to detect several objects simultaneously, instead of one at a time.
"One approach to this problem is to train, for example, five different detectors to recognize five different objects. But we want to train just one detector to do this. Developing that algorithm is the next challenge," said Vasconcelos.
Applications for the new cascade algorithm include autonomous vehicles, robotics, and image and video research systems.
For more information, visit the UC San Diego Jacobs School of Engineering website.