Vision-Based System Allows Humans to Work Safely with Massive Robots

The advent of the collaborative robot opened up a range of new applications, allowing humans and robots to work together in close proximity or even on the same tasks. Cobots also save floor space, as they don’t require an enclosed, fenced cell.

However, robots designed for safety and collaboration have a few drawbacks. They’re more expensive than non-collaborative robots of the same reach and payload. They can’t use non-collaborative end effectors with sharp edges or pinch points. They move slower and can carry lighter payloads than conventional robots. So, how can innovative, collaborative applications be found for larger and heavier payloads, such as white goods or in automotive assembly?

Patrick Sobalvarro is co-founder and CEO of Veo Robotics. Veo has developed a vision-based solution which creates a 3D point cloud of the area surrounding a robot, allowing the robot to detect when a worker enters the area. The robot can then slow or stop. With this technology, human-machine interaction is possible with ever larger robots and machinery.

Patrick Sobalvarro

Engineering.com recently sat down with Sobalvarro to learn more about this solution.

Why is human-robot collaboration important?

There are all sorts of ways that large industrial robots can be really useful in manufacturing, but they're not smart. In particular, they can't be ingenious the way humans can. Human production workers are very flexible, and they learn things really quickly. In an era where it is ever more important to respond to quality problems that might arise when you're manufacturing things, and people are getting product out at the market more and more quickly on shorter product cycles, and consumers are demanding more and more customization of the things that they buy. Think about a refrigerator—you can get it all kinds of coverings with different configurations, ice makers or not, cars are a similar sort of situation. You need the flexibility that a human production worker brings to a manufacturing process, but at the same time, we have an aging production worker labor force, and that's true in every developed and developing country.

The physical work associated with moving a refrigerator door or a car door or a car seat is out of reach for a lot of people in the production workforce. It’s necessary to help those folks with the kinds of things that robots are good at, and to make it safe for the people to be in there doing the things that require judgment and dexterity and great sensing like we have in our hands or our eyes.

What does Veo do?

What we do is we add that perception with our vision system and that intelligence, which comes from a highly redundant, highly reliable computer that runs test algorithms that we came up with. We make those robots sensitive to human presence in the same ways that humans can be when working together. 

Would you need a system integrator to set this up?

We partner with systems integrators and typically if you're a medium-sized business, you would buy through the systems integrator and they would come and install it as part of their work in setting up a process step on your line.

For example, if you're making stoves and they have a glass cooktop, and you need to dispense an adhesive that attaches the glass cooktop to the metal frame of the stove, you should hire a systems integrator who is an expert in that sort of thing. That systems integrator would install the cameras around the work cell as part of the general design of the work cell.

That work cell can also be not completely locked off from humans. A production worker could just walk in there if they needed to, in order to handle an exceptional case, or if it was part of the process themselves. For example, they needed to install some screws on the cooktop or something like that, they could just walk in, interrupting the cycle and the robot would slow or stop as necessary to keep them safe.

How many cameras would be needed and how would they be installed?

In a typical work cell, we see four of these sensors being installed. Our system is designed to automatically register those sensors to the base of the robot. So, you don't have to spend a lot of time figuring out how to point them and figure out the geometry of the work cell the way you would with an area scanner. Instead, the system will automatically figure out that geometry and bring all those images together into a single three-dimensional image. It's then processed to get the necessary commands to the robot so that the humans can be safe. If you have a highly occluded work cell with a bunch of fixtures in it or it's a very large work cell, you might install up to eight of them and you can connect all eight of them to the same processor.

Does the Veo system change the method of programming the robot?

The system doesn’t interfere with how people typically program their robots. We partner with all four of the biggest robot manufacturers in the world (Fanuc, ABB, Yaskawa, and KUKA). We work closely with those robot manufacturers to understand how we should interface with those robots. Then, the systems integrator programs the robot they way they typically would.

The system doesn’t try to understand the robot program. We allow the systems integrator, who's very trained and really knows how to make these robots do exactly what’s needed, to program that robot the way they did before. Our system overrides the robot controls to slow or stop the robot if necessary.

Can I use Veo with say a Nachi, Panasonic, another brand of robot?

Not right now. If you wanted to do that, we do have safety-graded discrete IO outputs on our systems. You could use it with one of those robots in the same way that you would use our system with say a stamping press or a CNC machine. Therefore, you can take those safety outputs and you can do something with the robot, but we don't directly control other robots besides the big four manufacturers. We started with the biggest four because in North America, most any industrial robots used in durable goods comes from one of those four manufacturers. The same is true for Europe and Japan.

Most robots have different stop categories, and E-stops or zero stops tend to cause more wear and tear than other stop conditions. How does Veo stop the robot?

Our system will only category zero stop robots if it detects a fault. The system is dual channel redundant throughout. The system is built in compliance with ISO 13849 and, as a result, our sensors have two separate imaging elements, two separate data paths and our processor box has two separate motherboards, two separate power supplies plus an outboard safety processor that's monitoring the entire system, and that safety processor is what communicates with the robot.

If power is removed, it immediately goes to category zero stop. Any fault condition will trigger a category zero stop, such as if a sensor is blocked or splashing something onto the sensor. Otherwise we never do a category zero stop. In normal operation, it executes a category one stop. The category one stop does remove power to the motor, but it keeps the robot on its planned trajectory. This stop takes between 0.5 and one second, depending on the robot model.

So, this system does not create additional wear and tear on the robot. We're very sensitive to that and we work closely with those robot manufacturers to make sure that the way we do things is in the best interest of the customer in terms of operating the robot.

For more on robot safety systems, check out Optimizing Efficiency and Safety with Intelligent Industrial Robot Cells.