Amazon has started deploying an innovative robot to handle the complex tasks needed to run the company’s sorting facilities.
Mobile robots are common in these facilities—Amazon already has more than 200,000 of them. But the company recently started installing stationary robotic arms as well to take on an important role in distributing packages to customers.
One robot in particular, named Robin, uses advanced technologies to take on far more complex tasks than other robots of its kind. These technologies include artificial intelligence algorithms that can make decisions almost instantly, high-definition cameras and sensors, and sensitive grippers.
Conventional robots often perform a single specific task, such as screwing bolts onto a component or welding vehicle parts together—repetitive tasks that can be performed identically. But Robin’s job is much more sophisticated, and tasks are rarely completed in precisely the same way.
Robin must manipulate and sort a variety of packages. To do this, it picks them up with a suction gripper, scans them, and places them on a mobile robot for transport to a designated loading dock. These packages vary extensively in shape, size and weight—so a programmed set of predetermined motions won’t do the job. Instead, Robin must adjust its actions in response to a constantly changing environment.
“Everything comes in a jumble of sizes and shapes, some on top of the other, some in the shadows,” said Bhavana Chandrashekhar, software development manager at Amazon Robotics, in a press release. “Sometimes, the differences between one package and another are hard to see, even for humans. You might have a white envelope on another white envelope, and both are crinkled so you can’t tell where one begins and the other ends.”
Robin uses a process called image segmentation to organize boxes, soft packages and letters coming down an assembly line. While people can perform this task automatically, it has taken a long time for robots to see more than just vague groupings of pixels. The Amazon Robotics team has developed a novel approach to teach Robin how to recognize the different packages.
Rather than uploading set computer vision algorithms to the robot’s brain to divide scenes into individual elements, the engineers used a pretrained computer model and gave it the leeway to examine an image to identify objects on its own. Once it selected an object, the team provided feedback on how accurate the model’s choice was. They started with simple object elements such as edges and planes and slowly trained Robin to make a more sophisticated analysis of what it was seeing coming down the assembly line.
Through this process, Robin slowly started learning to identify the many different packages it would need to handle. Robin is constantly being retrained using these images. Importantly, the robot also provides feedback on what it has learned and how confident it is about its decisions. For instance, if it marks an image as low-confidence, that image is automatically flagged for annotation and is added to the team’s training materials for reinforcement.
In addition, Robin has learned to recognize when it has made an error. For example, if it drops a package or puts more than one package on a mobile robot, Robin will attempt to correct its mistake on its own. If it is unable to, a human is called to step in and resolve the problem.
“Robin deals with a world where things are changing all around it. It understands what objects are there—different sized boxes, soft packages, envelopes on top of other envelopes—and decides which one it wants and grabs it,” said Charles Swan, senior manager of software development at Amazon Robotics and AI, in a press release. “It does all these things without a human scripting each move that it makes. What Robin does is not unusual in research. But it is unusual in production.”
The Robin model has only been deployed in a small number of facilities for now, as its operators continue to work on refining the robot’s accuracy. Training is still ongoing: the team uses new fleet metrics to retrain Robin every few days and aims to give it updates several times a week. Eventually the robot will be ready to deploy at scale, giving Amazon an innovative new tool to keep up with the high demands on its shipping and distribution centers.