A team of engineering graduate students at MIT’s Signal Kinetics group have developed a robotic system called FuseBot, which uses complex reasoning to identify and pick up objects. FuseBot can judge the probable location and orientation of objects within a pile of other objects, enabling the robot to successfully identify and retrieve any object, even if it’s hidden. According to the researchers, this technology could have applications in e-commerce warehouse operations by speeding up picking and sorting processes.
The FuseBot system builds on earlier work that used RFID tags to help a robotic arm locate hidden objects. However, there are challenges with relying solely on RFID tags. The retrieval process takes longer, and RFID technology can be expensive and inefficient in the long run.
The FuseBot team addressed these challenges by leveraging RFID technology to help it detect both RFID-tagged and untagged items. To achieve this, the technology introduced two key innovations that use radio frequency (RF) signals: RF-Visual Mapping and RF-Visual Extraction. These help FuseBot determine the most efficient way of getting rid of other objects that are obstructing the target item.
“If I give a human a pile of items to search, they will most likely remove the biggest item first to see what is underneath it. What the robot does is similar, but it also incorporates RFID information to make a more informed decision. It asks, ‘How much more will it understand about this pile if it removes this item from the surface?’” said Tara Boroushaki, a graduate student of electrical engineering and lead author on the paper detailing the FuseBot research, in an MIT news release.
RF-Visual Mapping is responsible for identifying and locating RF-tagged items. FuseBot uses this data to construct an occupancy distribution map which serves as a spatial representation of the object pile. Meanwhile, RF-Visual Extraction accounts for factors such as probabilistic occupancy distribution, expected grasp quality, and information gain from future actions. The latter was designed as an optimization problem to make the retrieval process much quicker, minimizing the number of actions needed to retrieve the target item from the pile.
To demonstrate FuseBot’s capabilities, the team used a robotic arm with a video camera and RF antenna attached to it to identify and retrieve untagged items from a mixed pile. The system scans the pile with the camera and uses the information to create a 3D model of the environment. Simultaneously, the antenna sends signals to detect the RFID tags. According to the team, FuseBot can easily determine that the item is not located in the same position as an object with an RFID-tag.
The algorithm updates the 3D environment according to all the information collected and uses this to pinpoint potential locations of the target item. The system then calculates the best possible retrieval strategy and reasons through the objects’ size and orientation in the pile, as well as the RFID tag locations to choose which objects to remove. The goal is to find the target item using the least number of moves.
“What this paper shows, for the first time, is that the mere presence of an RFID-tagged item in the environment makes it much easier for you to achieve other tasks in a more efficient manner,” said Fadel Adib, associate professor in the Department of Electrical Engineering and Computer Science and director of the Signal Kinetics group.
The team reportedly ran more than 180 experimental trials using robotic arms and piles of household items that varied in size. FuseBot reportedly achieved a 95 percent success rate and performed the retrieval process with 40 percent fewer moves than the previous system. It also located the target items much quicker.
“We see a big improvement in the success rate by incorporating this RF information. It was also exciting to see that we were able to match the performance of our previous system, and exceed it in scenarios where the target item didn’t have an RFID tag,” said Laura Dodds, an electrical engineering and computer science graduate who is part of the RFID Localization team in the Signal Kinetics group.
The research team is already planning to integrate more complex models to help FuseBot reason with deformable objects. They have also expressed their desire to work on different manipulations such as pushing items out of the way.