Unity Technologies has sponsored this post.

Consider the factory of the future. Will it look like a normal factory, only with everybody wearing silver clothing and scooting around with jet packs? Or will it look pitch black, because the lights are out and robots are taking care of everything while silver-suited humans lounge on the beach in jet-hammocks?

While it is difficult to see the future, it is somewhat easier to see the present, and the factories of today are in the midst of a significant transformation. The convergence of technologies such as artificial intelligence (AI), extended reality (XR) and real-time 3D platforms has given rise to incredible solutions for manufacturing—solutions which seem futuristic but are presently at hand.

Take one example: operator training. You’re building a new factory and you need to equip it with people who know how to run it. How do you train them? Well, you could get in the factory with them, take them through the procedure and supervise them as they try, again and again, until they master it.

Except, maybe there is a better way. Instead of waiting for the factory to be built, what if you could train your operators beforehand? You could create a 3D model of your factory and use virtual reality (VR) to host your training session. And instead of flying out to train your operators-to-be, wouldn’t it be easier if you could enter the VR factory from your office on the other side of the country? Instead of having to talk about hypotheticals—if there’s a toxic gas leak, then do this—you might as well program VR simulations of those hypotheticals, so your employees can acquire almost-real-world experience. They won’t feel the real heat of a VR factory fire, but they will feel the real pressure of the scenario, and you better believe they’ll remember it.

None of this is out of reach. To quote fictional engineer Oscar Goldman, we have the technology.

How AI, XR and Real-Time 3D Are Shaping Manufacturing

In engineering.com’s latest research report, The Factory of the Future: How AI, XR and Real-Time 3D Are Shaping Manufacturing, we take an in-depth look at the technologies that are disrupting manufacturing today. With real-world case studies and insight from manufacturing experts, the report uncovers just how much potential is waiting to be tapped for those willing to explore new solutions. Many manufacturers are already paving the way.

That operator training example? It wasn’t rhetorical. Ericsson, the prominent Swedish telecommunications provider, used virtual reality to train the employees of its new USA 5G Smart Factory in Lewisville, Texas. Before the factory opened its doors in March 2020, its future operators in Dallas were trained via a VR tour of a sister factory in Tallinn, Estonia, led by virtual avatars of the Estonian operators.

“We were able to provide our new employees with training for a physical environment that at the time didn’t even exist, without having to get them on a plane and fly them long distances, all because of the power of connectivity and virtual reality,” said Ericsson’s Anna Cau.

Example of collaborative VR assembly training. (Image courtesy of Light & Shadows.)

In the factory of the future, training itself may become obsolete. Improvements to human machine interfaces (HMIs), coupled with augmented reality (AR), could potentially build training directly into a factory itself. Think of a video game tutorial that teaches players while they play—the days are long past when anybody sits down to read a manual before booting up their Xbox.

“The end game is you just walk up and use it. You don't need training,” predicted Bart Manning, Senior Account Executive at Unity, a real-time 3D development platform. “The more of the information and training that you can present to people in a way that they can intuitively understand, the more you can move them through the learning journey by giving them an experience that is digestible by the human body and its muscle memory.”

Such solutions are made possible by extended reality (XR), a catch-all term for spatial computing technology including AR and VR. Real-time 3D platforms like Unity—sometimes known as game engines—are also an important component of such solutions, as they provide an environment for XR applications to be developed. Real-time 3D platforms are also an important environment for AI development, particularly for machine learning training, as we’ll see shortly.

Though AI, XR and real-time 3D platforms are only a subset of the technologies driving new manufacturing solutions, they are each a significant pillar on which these solutions rest.

Level 5 Industrial Robots

Let’s look at another example: industrial robots. Robots currently play a large role in manufacturing, from robot arms used for assembly to automated guided vehicles (AGVs) used for material transport and beyond. Industrial robots will continue to get more sophisticated. In the short term, emerging solutions are poised to make a real impact on factory efficiency by automating repetitive tasks that are just unstructured enough to stump current robots—for example, tasks such as sorting various items on an assembly line.

“Computer vision and other learning-based techniques are right on the cusp of unlocking this, and so there's a lot of opportunities for robotics to make an impact,” proclaimed Sarah Gibson, Senior Engineering Manager of Robotics at Unity.

One way Unity is used in this context is to generate training data for these robots. Machine learning training requires an enormous amount of data, and it must contain enough variety to encompass everything the robot may face. If you train your robot on data that contains only cubes, it will falter when it encounters a sphere. If you train your robot with cubes and spheres colored red, it will be perplexed by the color blue. You get the idea (much faster than a robot, which would need thousands more examples).

“If you need 100,000 images to train a model, it's much easier to get 99,000 simulated images and 1,000 real images than to get 100,000 real images,” Gibson explained.

Unity is used to create simulated training data for a robot arm. For robust training, the data encompasses a range of environmental conditions. (Image courtesy of Cross Compass.)

Ultimately, in the same way that engineers are developing fully autonomous vehicles (in SAE parlance, Level 5 autonomy), industrial robots will also approach complete autonomy. Such robots will adapt to changing circumstances and respond to given events without human intervention. One could imagine this extending to the entire supply chain, from raw materials to finished product.

“The long-term vision for Industry 4.0 is a completely automated economy,” Gibson said.

The Factory of the Future

AI, XR and real-time 3D platforms are unlocking real value for manufacturers today. Though technology will continue to improve, it’s not difficult to draw a straight line from cutting-edge implementations of these technologies today to what will be standard practice tomorrow.

We’ve just seen the tip of the iceberg of what AI, XR and real-time 3D can bring to manufacturing. In addition to upending operator training and industrial robotics, these technologies are impacting applications including factory design and layout, pre-commissioning, simulation and optimization, worker safety, monitoring and maintenance, factory re-configuration and upgrades, and quality assurance. The factory of the future will leave no gear unturned.

For much more information on the technologies impacting manufacturing today, read our comprehensive research report: The Factory of the Future: How AI, XR, and Real-Time 3D Are Shaping Manufacturing.