Individuals and businesses have had to adjust to the pandemic. Those lucky enough to have escaped the virus still have to deal with its effect. Gyms, restaurants, retail stores and salons have all needed to close, reopen, and make adjustments. Engineers faced with the pandemic have their own issues.
All the beliefs we took for granted about working at a business have been shaken—our methods and processes shattered. For example, not too long ago, if you had a problem you could not solve, you popped into your mentor’s office. If you wanted to know how the machinists can’t hold a 0.005-inch tolerance on a critical dimension, you paid a visit to the machine shop with the offending part in one hand and a digital caliper in the other. If you wanted to conduct a design review, you booked a conference room. Now that COVID-19 separates us from everyone we have to work with, what do we do?
You are miles from your office workstation, your peers, the machine shop, and your customers. But you, the engineer, are one of the lucky ones. You have had digital tools all along that let you work anywhere. That part was 3D modeled, prototyped, analyzed—perhaps even shared and reviewed online—in 3D. It may have taken a few months to get used to, but already a new normal has been established, a crossover from your old comfort zone, to working where you live. A grumbling acceptance gives way to all the advantages the new situation offers. Every day can be casual Friday. The commute has vanished, putting a chunk of time back in your life. Sure, it felt weird at first. But then, the deck was built, a backyard fort was built for the kids. You and your family have a new relationship. For better or worse, it varies case by case. But for certain, you have a new appreciation of your kids’ teachers. And the engineering…. It never suffered.
Emerging technologies that you might have thought were cool, yet bleeding edge, a little too busy for, or impractical now seem more practical—even necessary. Internet of Things (IoT) in your next product may preclude on-site maintenance and allow for remote monitoring. Virtual reality will let you immerse yourself in your design in a much safer way than visiting a construction site, factory floor or office. Augmented reality lets service and repair be done by someone at the plant rather than requiring you to travel there.
When you need to “visit” the construction site or the factory, or have a design review with your boss or customers, there’s technology that lets you do that safely.
The Design Review
A design review is where engineers on a project may see the first scale model or prototype. It often happens around a conference table. You miss that. You got to feel the prototype, prod it, bend it, measure it, criticize it. The model got passed around, eliciting comments and feedback. There’s nothing like holding a part in your hand, feeling its heft, the tactile response.
Now that the pandemic has robbed all of us of the design review, we are unable to see or touch the prototype. It is too much to bear and we are desperate for a solution. You try Zoom to show off a model. You hold it up to the webcam in your laptop and you see everyone on the meeting lean into their screens for a closer look. You hold the model closer to the camera, but the wide angle of the webcam distorts the model. It is not going well.
You resort to sharing your screen and showing a 3D CAD model, but that’s not the same. There has to be a better way.
You remember a technology that everyone was saying would be the next big thing—virtual reality (VR) and something like it… augmented reality (AR). It was interesting, had some potential, but you had to wear something on your head. Who could do that for long? Who could generate the VR content? Most CAD systems do not have a VR export. Even if they did, could you really hand out VR headsets to everyone? They were not cheap.
Too many questions and no easy paths to AR and VR appeared to us just a few months ago. Then COVID hit and we woke up to a world that had changed. A technology that once looked bleeding edge is starting to look a lot more practical. AR and VR were practically built for a pandemic—safe distance, no contact and no mask is no problem. And there’s a bonus: you don’t have to travel to the meeting.
For big corporations, traveling to meetings can be a huge expense. Apple, based in Cupertino, Calif., near San Francisco, manufactures its iPhones in China. Apple books $150 million worth of flights a year, or 50 business class seats a day according to information leaked by LAflyer and reported on by Fast Company in a January 2019 article. You can bet that with COVID-19, Apple has asked to have its executives and engineers make more use of FaceTime.
While the public’s adoption of video conferencing, once the domain of business, is an interesting phenomenon, it needs to be pointed out that simply showing a face with a voice is insufficient for all that engineers need. It works for sharing 3D models and results of simulation, of course, but for the instinctive need of an engineer to start sketching on whatever is handy—a napkin, notebook, tablecloth—Zoom cannot handle that. With Zoom, you cannot bend the part, feel the finish, get its heft, etc. It is what civilians call “kicking the tires.”
Used wisely, the 3D, immersive nature of augmented and virtual reality can offer an almost-there alternative. Think of Zoom but with 3D parts where your face would be. We, as engineers, are less concerned about showing our faces as we are about showing what we are working on. While those in other roles may be sweating the pandemic, such as the sales people who miss the handshakes, the banter warm up, or the machinists, for reasons previously mentioned, engineers can avail themselves of VR, a technology that offers itself in a time of need.
The use of AR/VR can also be useful on a big scale, such as in building construction, civil engineering and infrastructure projects. A consulting firm or architect has to overcome concerns of clients for their own safety and struggle to arrange meetings. With AR/VR, these concerns vanish.
AR/VR, the technology that was once novel, used to dazzle a client with high tech in the pre-COVID days, may now be described as a safe, practical and necessary way of doing business.
We went to leaders of the design software industry and asked them what they are offering as an AR or VR solution to life during the pandemic. What follows is what we heard back.
PTC, CEO Jim Heppelmann
Jim, what has PTC done in AR/VR in this time of need?
Digitalization saved the day for knowledge workers. Now AR brings digital to the frontline workforce, the 75 percent of workers who can’t work from home by definition.
Vuforia offers the frontline worker the equivalent of Zoom collaboration with Vuforia Chalk. You get YouTube “how-to” procedures with Vuforia Expert Captureand PDF document publishing with Vuforia Studio, but instead of displaying digital information on a flat 2D mode of a computer screen, Vuforia places it in the work environment where it is most impactful.
AR allows for the remote monitoring and optimization of human work. This is important because most factories and worksites run processes that involve both things and people.
I like to say that AR is “IoT for people.” An AR device like a phone or HoloLens converts the digital bits and bytes from the cloud to sounds and sights for the worker. And it works in both directions to monitor and optimize human work.
When you blend the concepts of AR and VR and IoT and AI, you get spatial computing. This is where it gets really exciting. Spatial computing gives you the ability to get the full power of the digital experience whether you are in the real-world worksite or experiencing it virtually from your office. Imagine being able to walk the floors of any factory from the convenience of your office. Plus, the entire experience is in real time and includes live data and analytics. This day is not far off, and we are paving the way there with the Vuforia Spatial Toolbox.
Let me add that AR transforms training and mentoring. Training changes from in-advance/just in case, to just in time, in full context. We can create a whole library of AR process knowledge that any worker can access at a moment’s notice. A novice gets trained when needed using the digital legacy captured from an expert. Plus, the digital expert can remotely look over their shoulder and coach them in real time. The costs savings and quality improvement opportunities are immense.
Jim, do you have any examples of PTC’s AR at work?
During the COVID crisis, Philips needed to add new shifts to meet ventilator demand. They used Vuforia Expert Capture to train new shift workers. They also trained remote workers in ventilator production at a new factory they were rushing to bring online in India.
Autodesk, CEO Andrew Anagnost
Andrew, what products/technologies does Autodesk have in the way of AR/VR?
|
|
| Andrew Anagnost, CEO of Autodesk (Image courtesy of Autodesk) |
We recognized years ago that AR, VR and mixed reality (MR) have the potential to change how our customers design, create and experience everything from factories, buildings and cars to training, learning and entertainment. Our customers use these technologies to transform 2D designs into interactive, immersive digital models, which gives spatial context to digital information. Especially now amidst COVID-19, our customers need to rethink client engagements, site visits, design reviews, training, etc., because we are trying to keep everyone safe and working efficiently in remote environments.
We’re partnering with leading technology companies such as Nvidia and Unity to accelerate AR and VR technology development in order to rapidly build and deliver value to our customers. In terms of Autodesk’s AR/VR/MR offerings, I’ll give you some examples.
Insite VR Integration with Autodesk Construction Cloud can reduce late-stage construction errors by thoroughly inspecting designs in immersive coordination meetings. Remote users can gather inside a 3D building information model (BIM), walk around, annotate, and revise models at any scale and from any perspective. Users have access to tools allowing them to draw, measure, and add notes using speech-to-text. BIM metadata is available in the virtual environment, so users can easily inspect element details.
Our software helps automotive companies and startups in the conceptual design phase, where future mobility products are envisioned and visualized, [such as] self-driving or autonomous vehicles. The AR and VR workflows in products like Alias and VRED help our customers to better create forms and shapes, understand proportions in the context of their design ideas, empower collaborations and digital decision making. With VRED VR, we have customers host collaborative VR review sessions making digital decisions together in one virtual space.
Siemens Software, CEO Tony Hemmelgarn
Tony, what products/technologies does Siemens have in the way of AR/VR and how can they help during the pandemic?
Tony Hemmelgarn, CEO of Siemens Digital Industries Software. (Image courtesy of Siemens Software.)
Today, Siemen’s VR takes visualizing the digital twin to the next level by enabling immersion into a collaborative, full-scale 3D virtual environment and Siemens AR enhances the physical world with digital information. This dramatically increases understanding and eliminates design ambiguity, which reduces design flaws and process errors—ultimately contributing to higher quality products delivered to the market in less time.
All of our VR solutions are collaborative. Of course, collaboration has always been important, but it’s especially relevant now because of COVID-19. We’ve worked hard to make VR collaboration easy. Think about a Zoom meeting in 3D space where all the participants are able to simultaneously interact with the products and manufacturing facilities. This includes early concept and design reviews, human factors simulations, factory layouts, production line simulations and CAE simulation reviews. This is a very powerful, cost-effective capability that increases productivity, eliminates travel expenses, and enables everyone to remain safely at home.
Each step of the visualization evolution has made it easier to understand complex products and production systems. VR enables higher levels of fidelity and utility than ever before because it enables realistic interaction with a 3D virtual product. This means that the user can interact with and manipulate the virtual product as if it were a real physical product. Interacting with objects at true scale is the best way to understand them and improve their form and function. With VR, the digital twin can now be used for human factors evaluation and ergonomic analyses that previously required a full-scale physical prototype.
For example, using VR an automotive engineer can sit in any seat in the car and evaluate sight lines, HVAC flow and cabin ergonomics. This is impossible using traditional CAD tools with a mouse and keyboard and was previously accomplished using a costly physical prototype.
We believe that creating a comprehensive digital twin supported by closed-loop feedback of performance data is the key to digital transformation. Our Xcelerator portfolio includes the tools necessary to create and visualize a comprehensive digital twin that spans the entire product and production lifecycle. Without visualization technologies like VR, understanding the digital twin is extremely time consuming and requires substantial effort to interpret, analyze and utilize digital models and data. Visualization is critical to rapid understanding.
Our strategy is to offer VR and AR solutions tailored for every phase of the lifecycle. In other words, we offer the right visualization at the right time. Our solutions enable collaborative, realistic interaction with the comprehensive digital twin at any scale, in any environment. Each solution is seamlessly integrated into our Xcelerator portfolio and offers effortless access without data preparation as well as lifecycle-specific features and interactions. Integrated access without data preparation means that Siemens Virtual Reality is now practical for highly efficient work-in-process design and engineering workflows, design reviews and training scenarios for assembly and maintenance.
Xcelerator includes several AR and VR solutions. For example, we have NX virtual reality products take an NX 3D model and create an immersive, full-scale 3D environment with a single button click. This makes it practical for work-in-process design and engineering workflows as well as design reviews. Intuitive controls consistent with the NX user experience paradigms make it easy for users to rapidly move between the CAD and virtual worlds and improve the design.
NX VR enables visualization of the digital twin at full scale or larger than life to facilitate natural and close inspection of details in a way that standard desktop visualization cannot deliver. NX multiuser VR sessions include spatial audio integration to take design review participants to the next level of immersion and collaboration. Each collaborator has their own avatar and is visible to other participants. Collaborators can be located across different geographical locations for maximum efficiency, reduced travel costs and personal safety.
We offer STAR CCM+ Virtual Reality for immersion into CAE simulation environments. With STAR CCM+ VR, our customers are able to explore and interrogate results anywhere in the model at any point in the simulation. This would be extremely tedious using a mouse, keyboard and traditional reports. Using STAR CCM+ VR they can fly anywhere in the flow field. STAR CCM+ VR enables companies to safely explore, understand and optimize products, even those that operate in physically in hospitable or unsafe environments.
Our Process Simulate Virtual Reality enables the review and modification of process plans in an immersive 3D environment with a single button click. It provides the ability to experience the plant layout in full scale, run realistic process simulations, provide operator training, and resolve issues before going to production. It’s especially important for plant safety.
Let’s say you are placing safety gates to prevent a person from being trapped in a work cell when it’s energized. The gate location should be in a place along the fence line that provides a clear unobstructed view into the cell. This requires a perspective from the gate looking into a cell and is an ideal application of VR technology.
Teamcenter Visualization VR takes users into an immersive 3D environment using their JT [file format] models. Using familiar Teamcenter Visualization navigation features from within the immersive environment, users can view and analyze the 3D model at the proper scale, enabling human factors and ergonomic analyses that would be impossible in a 2D environment. In addition, VR increases the clarity of traditional digital mockup operations like interference and clearance measurements, sectioning, positioning and markup.
Our AR product is AssistAR. It is a solution for electronic work instructions designed specifically for complex assembly instructions. It takes the digital parts and assembly sequencing information (BOP—bill of process) from Teamcenter and applies these to the physical assembly. It reduces assembly time, improves quality, and reduces operator training requirements.
Manual assembly procedures that gain the most value from using AssistAR as a work instructions solution include complex and long procedures. For example, assembling the airframe of an airplane takes several hours and involves several hundred parts. Even experienced airframe assemblers need to follow detailed work instructions documents to successfully complete the job because the process is complex and long. With AssistAR, the airframe assemblers can quickly figure out what to assemble in the current step, and easily follow the assembly sequence for the rest of the process.
All of our AR and VR products integrated with Xcelerator are easily accessed and require no manual data preparation. All VR solutions offer multiuser same session collaboration in the virtual environment. All AR and VR data are managed in Teamcenter for reuse, accountability and to maintain the integrity of the digital data thread.
Are there customers that have made great use of AR and/or VR?
I would like to recognize one of our customers, BAR Technologies, for their extensive deployment of our visualization technologies, including VR. BAR is a company that innovates new maritime products and technologies. They were challenged to develop a superior crew transfer vessel for taking technicians to work on offshore wind turbines. They wanted to improve passenger and crew comfort and safety while dramatically reducing fuel requirements. The resultant design is a craft that is truly novel, reduces vertical accelerations by between 35 percent and 70 percent and is between 30 percent and 50 percent more fuel efficient.
To design the new vessel, BAR made extensive use of Xcelerator to create a comprehensive digital twin. They used NX Virtual Reality to design and review the crew transfer vessel at full scale. This provided BAR with a unique perspective that helped them check accessibility, test sight lines, and review other critical engineering challenges in a fraction of the time it would have taken with traditional methods.
Trimble, Senior Director of Emerging Technologies, Aviad Almagor
Aviad Almagor, senior director, Emerging Technologies, Trimble. (Image courtesy of Trimble)
Trimble’s portfolio of mixed reality (MR) solutions includes wearable see-through devices, hand held devices and an integrated suite of edge and cloud-based software solutions. Our MR portfolio is integrated with Trimble GNSS, BIM and cloud technologies to enable in-context visualization of 3D content and a smooth end-to-end workflow. Connecting the digital and the physical world is the primary driver of improved productivity, quality and safety.
Mixed reality has proven to be a viable solution during the pandemic, supporting remote collaboration and enabling remote monitoring of on-site activities.
During the design stage, project stakeholders use Trimble Connect for HoloLens for remote collaboration. The application enables real-time sharing of a 3D immersive experience with multiple users. In this immersive environment, project teams can see each other as avatars, communicate in realtime, solve design issues, and perform design coordination from the comfort and safety of their home or home office.
[Trimble Connect surpassed the 10-million user mark in June, according to Trimble’s recent press release, with 1.2 million of users signing on in March and April, a surge that can be attributed to COVID-19, Ed.]
For on-site teams, Trimble XR10, the only HoloLens 2 solution certified for use in safety-controlled environments, enables visual communication of on-site issues and streaming of a hybrid of physical/digital information to the office team.
Using the Trimble XR10 together with Microsoft Dynamics 365 Remote Assist application, experts have the ability to be connected to team members anywhere in the world, and effectively monitor multiple construction sites, spatially mark issues and instructions, and guide on-site team members from their home.
For outdoor use, Trimble SiteVision enables on-site teams to place and view georeferenced 3D models in context, including subsurface infrastructure. Equipped with Trimble Catalyst and EDM technology, SiteVision provides highly accurate, in-context visualization and a visual way to explore, understand and communicate complex information. On-site users can stream data to the office team, document site conditions, and consult with remote experts, reducing the need for site visits and ensuring the safety of the team.
Dassault Systèmes, CEO of Dassault Systèmes SOLIDWORKS Gian Paolo Bassi
Gian Paolo Bassi, CEO at DS SOLIDWORKS. (Image courtesy of Dassault Systèmes.)
This thinking is guiding our strategy in the field of AR/VR. We call AR/VR the immersive virtuality to better reflect the evolving continuum of technology and devices that today connect real to virtual.
The 3DEXPERIENCE Platform is designed to support a very large set of devices that are entering the immersive virtuality market, and we have released several applications to help our customers create the experiences they value for their business.
The use cases that the 3DEXPERIENCE platform supports span from the ideation all the way through engineering, manufacturing, marketing and operation of a product or service.
We have six categories of immersive virtuality applications.
Let me illustrate each of them, and the related business outcomes that make immersive virtuality so compelling.
- User experience design. This includes applications from the ergonomics of an assembly line to the layout of controls inside a car. Full virtual reality is the best approach here. It reduces the number of prototypes and makes it easy to get feedback from all possible stakeholders. It should be used especially in the ideation phase.
- Understanding complexity. Includes applications for multidiscipline reviews, analysis of complex simulation scenarios or virtual navigation of entire plants, existing or planned cities. Augmented reality allows superimposing metadata or computational analysis to real objects to reveal what is not obvious or only available in corporate databases. It allows faster decisions and reduces errors.
- Learning and education. Applications reduce the cost and the risks of physical training for many workflows from machine operations and maintenance to emergency interventions. A completely new field is also emerging to study, conserve and democratize the access to the world cultural heritage through immersive virtuality.
- Augmented operations. Applications can support operators in the field, reducing cognitive load and stress while increasing safety and quality. Detailed instructions can be delivered in real time to operators of any skill level, anywhere. An interesting emerging application field is life science with virtual planning of complex surgery made also possible by the rapid digitalization of a patient’s body organs.
- Brand experience. There are endless opportunities to use immersive virtuality to launch new products with amazing immersive experiences that capture attention and create a “wow” factor, so essential to get a message through and increase both online and in-store attractiveness.
- Sales experience. Applications like sales configurators and virtual tryouts can achieve greater customer involvement, gain trust and loyalty, facilitate the purchase decision of items that would otherwise need an in-person inspection or a leap of imagination.
We have released several 3DEXPERIENCE “roles” that support one or more of these categories of applications and are immersive virtuality-ready:
- Product Experience Presenter
- Immersive Visual Experience
- Immersive Collaboration Experience
- Immersive Ergonomics
- Marketing Experience Artist, Scripter, Reviewer
- Applications like Natural Sketch and HomeByMe are also adopting the use of immersive virtuality for freehand sketching and interior design
Gian Paolo, what applications are using SOLIDWORKS AR and VR?
SOLIDWORKS has been supporting VR since the release of Visualize 2018. The use cases in rising demand due to the shift to remote work are multiusers collaborative design review, training and post-sales services. A large ecosystem of partners is working directly with customers on specific and proprietary workflows. Here is a list of partners with interesting applications: SimInsights, Cavrnus, Mindesk, Sphere, Exxar, Masters of Pie.
At Dassault Systèmes, we think that a virtual twin is a new and more advanced way of representing things and life itself. We think that a virtual twin will help better imagine and balance the possibilities of industries with the reality and demands of societies and human beings.