Augmented Reality (AR) has been touted for many product development roles, from concept visualization to design reviews to immersive “brochures.” However, PTC is one software vendor who is betting that Maintenance, Repair and Operations (MRO) will be the killer app for AR. If your company makes equipment or is in the business of installation or maintenance, things are about to get interesting.
To be clear, AR is not the same as Virtual Reality (VR). VR recreates a world in 3D and often requires such a high level of graphics processing that users must remain tethered to a workstation. AR is less demanding. It generates less imagery and superimposes this imagery onto a real life view. It can be powered by as little as a smartphone. This makes AR much more accessible and economically viable for field service.
You can try AR for yourself by downloading the Vuforia app and pointing it at the “Vumark” code further down in this story.
Scroll down if you want to see how AR looks on your own iOS smartphone or tablet.
Why Caterpillar Generators Use Augmented Reality
A Caterpillar repair technician uses an iPad to see AR information displayed over a generator.
According to Terri Lewis, digital technology director of energy and transportation for Caterpillar, AR on an iPad or tablet is a far superior way to deliver how-to information when compared to a paper manual. New users in particular will find it easier to point an iPad at the generator to see how to operate it.
Caterpillar is exploring how to take the AR experience one step further with animations. These animations can guide service technicians to provide a specific maintenance step. To initiate the animation, a technician can view the generator through an iPad that is loaded with an AR file.
Caterpillar demonstrated this functionality in June 2016 by showing how to replace a fuel filter using an animated rendering that was overlaid on the physical generator.
The Combination of AR + IoT Is Much More Powerful
By connecting these generators to the Internet, Caterpillar can enable its dealers and service technicians to monitor rental devices in the field from a central location. The operations dashboard can tell where the assets are, whether they are running or on stand-by and whether they are operational or under maintenance. The technician can even operate the device by remote control through an app.
Caterpillar’s generator was loaded with sensors that connect to an Internet of Things (IoT) platform called ThingWorx. The demonstrative AR experience included animated gauges that displayed results that were gathered from the sensors such as:
- Fuel remaining
- Pressure sensors
- Electrical output
This is useful for monitoring asset utilization and for planning maintenance.
Whether or not AR is right for your product depends on whether maintenance is complex enough that imagery overlaid on the physical object would be useful. Most industrial equipment and consumer appliances would fall into that category.
An Augmented Reality Demo
To test-drive this technology, you will first need to download PTC’s Vuforia app and point your smartphone at an identification mark that identifies the physical object, much like a QR code. Vuforia calls these coded tags “Vumarks.”
When you point your newly AR-enabled smartphone or tablet at a Vumark, it loads and displays the AR graphical data. If you have never seen AR in action, this will be a “wow” moment. Your phone will display a 3D image of a simplified CAD model, or whatever graphical data the developers want you to see.
If the equipment you are viewing in AR has sensors and is connected to the IoT, your phone can display the sensor data to show you the status of the equipment in real time.
To test the user experience, take the following steps with your iOS smartphone or tablet:
- Visit the Apple app store.
- Download Vuforia from PTC
- Launch the app
- Point your tablet or phone at a “Vumark” like the one below. It works best if you can lay your target screen (in my case a laptop) flat since the orientation of the model is perpendicular to the Vumark.
You should see a navigation button to launch a Welcome Experience. Select this button and you will see a 3D model of a motorcycle that you can navigate by moving your iOS device around the Vumark. When you tap the right-side navigation button you can take several additional steps such as exploding the model or viewing sensor data.
The Technology Building Blocks of Augmented Reality for MRO
The Caterpillar MRO example was based on technology available from PTC. The technology building blocks for the AR it demonstrated are:
- Pre-existing CAD models
- A CAD authoring tool
- An animation tool if animation is required
- A publishing tool for Vumarks
- Software that controls what is displayed on-screen
If you also want to access the additional value that comes from publishing sensor data into your AR application, things will get a lot more complicated. You will need to go through all of the steps of designing your product for IoT such as adding sensors, ensuring security and deciding where the data processing will occur.
PTC claims to have understood this challenge and responded with a range of recently acquired solutions. The company’s buying binge has included Coldlight for analytics , Kepware IoT connections , ThingWorx for IoT control and Vuforia for AR among others. It is now undertaking the difficult yet important task of integrating these into a common platform.
According to Rob Gremley, group president at PTC, “A platform makes it faster and easier to create solutions,” so the company is working toward integrations that will allow data to flow from CAD tools to its IoT and AR solutions. Recent announcements highlight this path toward integration with the latest being the launch of “Vuforia Enterprise,” an application that can import a CAD drawing and turn it into an AR object.
PTC EVP Mike Campbell provided an overview of the technology that PTC used to make the AR demonstration of the Caterpillar generator.
Campbell started with pre-existing CAD models and then imported that data into Creo Illustrator to animate it. He then launched the Vuforia Studio application, which looks like a traditional 3D model space. Vuforia matches the 3D animation to the AR model. A user interface which comes with the AR model allows the user to control what he or she sees.
To take AR applications onto the IoT, Campbell demonstrated how Vuforia Studio can place virtual gauges to display data from the real sensors on the generator. These virtual gauges represent a view into ThingWorx, the software that processes and manages the sensor data and pushes the data out to the AR application.
Will Users Find Value in AR for Products?
“There will be cultural hurdles for sure,” said Jay Wright, president of Vuforia. However, unlike many enterprise software implementations, AR should have the benefit of eliminating paper.
That said, the user experience of using a tablet or smartphone is not ideal. Wright acknowledged that point, saying, “We see a device on your head as being superior to a device in your hands.” However, that is not likely to be the way most users experience AR for the first time.
The good thing about tablets and phones is that the majority of prospective users already know how to operate them. The same cannot be said of head-mounted AR and VR devices.
Michael Blumberg, a service industry analyst, commented that, “While MRO teams will quickly grasp the value of AR from an intellectual perspective, the adoption will likely occur gradually. Implementation of AR requires MRO teams to modify their workflow and perceptions of themselves as problem solvers. Many MRO teams have been conditioned to rely on their own experience, intuition and tribal knowledge to resolve problems. AR changes that basic premise.”
Michael went on to say, “Furthermore, they’ll have to remember to bring their tablet computers or AR glasses into the field with them, activate their AR applications and rely on the information that is presented to them to complete the task at hand. These activities are not second-nature and may take some getting used to for veteran MRO technicians.”
Terri Lewis and the Caterpillar XQ35 generator.
If you are curious about what AR can do for your field service teams, Vuforia looks like a relatively easy path for testing. Pricing is set using a “freemium” model so developers can create a proof-of-concept without significant cost.