What Is Augmented Reality and How Can Engineers and Designers Use It?

The term“virtual reality” has become fairly well known. If you've ever seen any of the Star Trek TV series or movies, then you’re probably familiar with the holodeck on the Starship Enterprise. The holodeck, also known as a holographic environment simulator, is an empty room equipped with a “hologrid” and the ability to produce holographic projections that are used to create any form of environment the user desires—environments used for everything from entertainment, recreating in familiar places, participating in interactive sports, practicing skills, training simulations, investigative purposes and more. These environments may seem real to the user, but obviously they are not. They are an entirely simulated “virtual” reality.

While at the present time the holodeck is just science fiction, virtual reality is currently being used in many industries today. The gaming industry is one example. Virtual reality allows gamers to experience being in a three-dimensional environment and interacts with that environment and other players during a game. While virtual reality definitely has its place in a myriad of different industries, what about the possibility of combing some of these virtual realities with real-life situations? An enhanced reality where the user can see and experience the real world around them, but with the addition of computer-generated images and objects in conjunction with what’s really there: enter augmented reality (AR).

While AR may be a lesser-known term, it is rapidly gaining more exposure. It has a fairly long history and has reached across many industries. In this two-part series, I’ll take a look at the history of AR and what current and potential uses it has in the computer-aided design (CAD) industry and, in a later article, in the building information modeling (BIM) industry.

First let’s take a look at the history of AR. Its beginnings can be traced all the way back to the 1950s and 1960s to Morton Heilig who is now referred to as the “father of virtual reality.” Morton was a philosopher, film maker and inventor who applied his cinematographer experience to design and then finally patented the Sensorama Stimulator in 1962. The Sensorama Stimulator used visual images, sounds, fans, scents and vibrations to give the user the sensation of riding a motorcycle down the streets of Brooklyn. Albeit big and bulky, the Sensorama Stimulator was ahead of its time. Unfortunately, Morton was unable to obtain the financial backing necessary to move the invention ahead any farther.

One of the next big milestones in the history of AR was the invention of the first head-mounted AR machine. In 1968, Harvard associate professor Ivan Sutherland worked with his student Bob Sproull to invent what Sutherland termed, “the ultimate display.” The system displayed a simple wireframe room that was shown through the head-mounted binocular, and the user’s perspective from within that room would change using vision and head tracking. While the user interface may have been head mounted, the entire system was so large and heavy that it could not be placed on the user. It was suspended from the ceiling directly above the user. This inspired the name that was given to the device, “The Sword of Damocles.”

Even though we now refer to these early incarnations as AR, it wasn't until 1990 when Tom Caudell, a researcher at Boeing, coined the term “augmented reality.” Caudell was working on an alternative to the large bulky “example” boards that were used then as a guide for factory workers installing the wiring on planes. Instead of having to build an example board that installers would follow for each plane and process, Caudell and fellow co-worker proposed a head-mounted apparatus that would be used to project the wiring schematics or instructions onto a special reusable board. These AR displays could quickly and easily be changed with a computer, as opposed to having to manually reconfigure or rebuild the board by hand.

Later that decade, around 1998, AR made one of its first mainstream debuts. The application was created for televised football games. A first-down line was superimposed over the actual view of the football field for people watching on TV. Then we started to see the technology being used in weather forecasts, overlaying computer images onto real-world views and maps. From there, developments in AR really started to explode. In 2000, while in the Wearable Computer Lab at the University of Southern Australia, Bruce H. Thomas created the first mobile, outdoor AR game called ARQuake. Around 2008, smartphones started to use AR in applications such as maps. Then in 2013, Google released Google Glass, which is a head-mounted optical display that resembles a pair of glasses. And just last year in 2015, Microsoft released the HoloLens—an AR headset that blends computer-generated images (holograms) with the real world around you.

The revenue potential seems to be growing just as fast (if not faster) than the developments themselves. While AR revenue is expected to reach $1 billion this year, it’s forecasted at $120 billion by 2020. AR has become far reaching, with applications spanning across a myriad of industries. Here are just a few examples:

• Archaeology: Display ancient ruins as they looked at a particular site the way they existed in history.
• Art: Help individuals with disabilities create art by tracking eye movement and turning those movements into drawings on a screen.
• Commerce: Show multiple customization options or additional information for a product.
• Education: Superimpose text, graphics, video and audio onto a student’s real-time environment.
• Fashion: Show what different makeup or hairstyle options might look like on a person.
• Gaming: Allow users to experience and interact with a game using a real-world environment.
• Medical: Show patients’ internal organs superimposed over their skin via virtual X-rays.
• Military: Use AR goggles in real time to show people and various objects and mark them with special informative indicators and to warn soldiers of potential dangers.
• Navigation: Label road and street names along with other pertinent information on a real-world map or display on your wind shield showing destination direction, weather, terrain, road conditions and traffic information as well as alerts to potential hazards.
• Sports: Show the first-down line on football games, the flight of a golf ball after it’s hit or puck tracking during hockey games.
• Television: Display weather visualizations and images.

So it is evident that AR has a rich history, has a lot of revenue potential and touches almost every industry. But how is it affecting the architectural BIM and manufacturing CAD industries and what things might we have to look forward to in those industries in regards to AR? Let’s take a look at manufacturing and CAD first.

One of the main advantages AR brings to CAD is the ability to show a design that exists solely inside of your computer’s design software, as it is meant to appear in the real world. Imagine a machine operator or someone in the assembly department using a tablet to superimpose guided process instructions or a view of the final assembled product to follow during the assembly process. Within the last few years, this has all become possible with the appropriate CAD software, 3D data from that software, a smartphone or tablet with a camera and the right amount of computing power. With this technology, a target is used in the real world to keep the augmented 3D data in the correct placement of the real world. The target usually takes the form of a printed QR code. The target syncs the location of the digital 3D data with the real world you are viewing, allowing you to walk around the object and look up or down on the object just as you would if it was there in real life.

HMDs

Recent advancements in technology have added an additional option for AR viewing above and beyond using a smartphone or tablet. Head-mounted displays (HMDs) can be modified to utilize see-through glasses or goggles that can project images over what you are looking at in real life. Here are a few of the top examples of HMDs:

• Microsoft HoloLens: This holographic computer enables you to interact with high‑definition holograms in your world.
• Oculus Rift: Oculus Rift uses state-of-the-art displays and optics to provide incredible visual fidelity and an immersive, wide field of view. Technically a VR unit, it can be modified for AR.
• ODG R-7 Smartglasses: This advanced platform is designed for AR head-worn experiences

All of these hardware options have applications in the world of CAD by allowing the user to see 3D digital data, as it would appear in the real world. And for most companies or individuals working in this field, digital data already exists or can easily be created in the form of 3D models from their CAD software.

As I mentioned above, one of the biggest advantages AR brings to CAD is visualization—allowing you to see a finished design in the real world, even though it has not been built yet. In addition to visualization, AR has applications when it comes to the actual manufacturing process as well, such as quality control inspections, maintenance and training. For a real-world example of how these applications can help in the design industry, we look to Dexter Lilley, executive vice president and COO at Index AR Solutions. “On the third AR project completed by our team, which was a complex structural product with almost 1,000 welds, there was a 35 percent reduction in both schedule and cost with zero-quality escapes,” he explained. “More recently, on a complex inspection job, we saw a 90 percent reduction in cost and schedule. In fact, we found that the inspection function lends itself extremely well to augmentation where the 3D engineering model is available,” he added.

So what are CAD software providers and others in the industry doing to take advantage of and incorporate AR? Let’s take a look.

SOLIDWORKS has AR available in both the standard and professional versions of eDrawings for iPad and iPhone. You can show your design as it would appear, full scale, in the environment that it is meant to be in, so customers or other members on your team can see exactly what the end result will look like. You can even upload a full product line into eDrawings and use AR in conjunction with configurations to show specific product variations to a potential customer. eDrawings Standard and eDrawings Professional are both available for download at the App Store.

Autodesk at one point had an AR plug-in for Autodesk Showcase. It was a pilot project from Autodesk Labs, but the project has since been retired. In 2015, however, they announced plans to make Autodesk 3D models compatible with Microsoft’s HoloLens. A press release on the subject stated that digital models using software such as Autodesk Maya or Fusion 360 could then be viewable in the mixed-reality environment enabled by Microsoft’s HoloLens.

PTC, maker of the CAD software PTC Creo, has entered the AR market by acquiring the AR platform Vuforia. Jay Wright, Vuforia general manager and senior vice president of PTC, stated, “Our goal with Vuforia is to deliver an AR experience on top of all types of things—and fundamentally change the future of work.” The Vuforia platform supports Android and iOS smartphones and tablets as well as digital eye wear.

There are also some third-party AR providers that have software and applications for download. Augment is one of those with apps available for both Android and iOS platforms. Augment is labeled as a user-friendly turnkey solution that you can upload your 3D models to, and if you don’t have your own 3D models, they have an in-house team that can create them for you or you can browse their gallery of pre-made 3D models.

AR can be used in almost every phase of design and manufacturing, from the initial concept and design phase, in which you are reviewing and evaluating concepts and alternatives as they would appear in the real world, to the manufacturing phase, in which process steps, assembly examples and quality control information can be superimposed into an actual work environment. AR can provide detailed information on maintenance procedures, overlaying them directly onto your part or machine. You can even use AR in marketing your product or design, showing different configuration options and how the product might look in a client’s setting.

The product design phase is the phase where AR affects engineers and designers the most. AR can allow an engineer or designer to design a product right in the environment it will go in, allowing them to be aware of space restrictions or other obstacles. It could also allow you to work aesthetics into your design, making sure that your product will look pleasing to the eye when it’s finished and installed. Or perhaps you need to design an addition to an existing part or machine but you don’t have drawings or models of the existing item. AR allows you to design directly on the existing item, taking into account size, shape and existing features.