Life Post-Pandemic: How Mixed Reality Is Changing the Way We Entertain

Will “social distancing technology” become the norm post-pandemic? In the first part of our two-part series, Denise E. White, the founder of SPATIALx as well as flagship brands BLANK XR and Immortal XR, said that might be the case. According to White, her companies are leading us to a new technological era by using mixed reality in retail and entertainment spaces. Delving into real-life examples, White explains the role of mixed reality and the challenges of designing for a mixed medium life post-pandemic.

Decoding the Design with Guest Denise E. White

Watch our interview with the founder and CEO of BLANK XR and Immortal XR Denise E. White. For full details, read the article below.

Trends of the Post-Pandemic Digital Transformation

According to IBM’s 2020 U.S Retail Index, the pandemic sped up the shift to digital shopping by about five years. While lives have gotten busier and COVID-19 regulations have made it harder for people to feel connected, immersive and digital experiences will soon take their place.

A recent NielsenIQ global survey found that about 60 percent of respondents said they were turning to technology to simplify their hyper-lives as they got busier. Examples included virtual experiences to preview furniture and products from brands like IKEA and Home Depot or to try on fashions from Louis Vuitton and Gucci.

While some companies and universities aim to conduct everything remotely, many are shifting to build a digital work or school culture. According to Microsoft, mixed reality immersive experiences can lead to a 22 percent improvement in test scores for post-secondary students and a 35 percent increase in student engagement and retention. Mixed reality can also provide opportunities for students to learn and train simultaneously by enabling works with virtual exposure to different scenarios that would typically be difficult to recreate in real life. Medical students can also use holograms to practice procedures or operations. Mixed reality can help save costs, increase learner productivity and increase safety.

We’ve seen a bigger digital transformation in the last six months than we’ve seen over the past decade. However, this is only the tip of the iceberg. Similar technology is driving consumers to specific retail and entertainment locations to increase purchases and for music venues to enhance live performances or to allow consumers to experience them in the comfort of their own homes.

Mixed Reality: Playing Live in the Digital World

Using a combination of human-computer interaction such as voice and skeletal tracking, computer inputs and environmental inputs, mixed reality has the power to place digital content in the real world and hide the physical one. Many companies have started creating virtual live entertainment, including Sensorium, which has teamed up with Mubert to create a new-artificial intelligence (AI)-based DJ that jams out at virtual concerts, as well as WaveXR, which has already partnered with stars like Lindsey Stirling, Tinashe, and John Legend.

Immortal XR allows fans to converse directly with their favorite artists in a holographic form 24/7 through digital twins. Using AI and machine learning, an artist will know a fan’s name, answer any questions, and even sing their latest single, enabling each conversation to be personalized. The company relies on 108 HD 5K cameras that capture a person from all angles. Using a digital twin platform and AI, the company can successfully manipulate video and clone an artist’s voice to make it seem like the viewer is talking to the performer.

“You might think to yourself, There’s a great band that I love and my friends and I will go down to the venue and we’ll see it in person. But that’s [only] one way to experience that particular band. But that band next month might be having a virtual event and maybe your friends from around a different part of the world might want to attend that show, but you wouldn’t be able to otherwise attend with that particular person unless you went to a virtual event together,” said White. “It just becomes a different type of experience that you would attend for different reasons. We’re offering more choice into the ecosystem of events and experiences,” she noted.

Mixed Reality: Coming to a “Store” Near You

While many physical stores are now reopening, stores are keeping safety as a top priority and are turning to digital means to achieve this. In fact, expected to be launched at the end of 2021, VR Pay will allow virtual reality (VR) shoppers to browse through VR shops and malls. This can augment the consumer journey and extend real-world access to online, in-store, on the go and new markets.

Augmented shopping enables customers to engage with brands and products via a digital experience. For example, BLANK XR allows shoppers to find the right merchandise by holographically projecting products through one’s biometric data profile. Using AI and machine learning, these experiences can help deliver more detailed product information than standard web experiences, enabling customers to feel more confident in their product selection.

(Image courtesy of Deloite Insights.)

“What we’re trying to do is actually enhance that real-life experience, and so it becomes another choice for the consumer to have.… One of the things that we are really passionate about is creating an experience that’s much more centered around mixed reality than it is virtual.… What we’re trying to do and aiming to do, is to create a much more realistic feel, so it actually looks like the actual artist,” explained White. “We are in the throngs of development, but we’re nearly there, and for us it’s about creating something that feels believable.”

To expand 3D graphics, many organizations are addressing key challenges with rendering 3D products, including augmented reality (AR) shopping. The Khronos Group created the WebGL standard for accelerated 3D graphics, which helps provide an industry foundation for any application to download and render 3D products. The three leading processes include photogrammetry (it combines overlapping pictures to reconstruct data to make a single model), 3D scanning (it scans an object’s shape and measurement to create a vector point cloud), and 3D modeling (a model created by a 3D artist).

(Image courtesy of Deloitte Insights.)

Augmented shopping involves much more than just bringing something digital into a physical experience and is about seeing the materials and details of a product such as its craftmanship. Many companies have started to pick up on these trends, such as L’Oréal, which enabled customers to imagine a new look or color on themselves. The company doubled its website engagement and tripled its customer conversions. Furniture stores like IKEA and automotive brands like Toyota have allowed customers to virtually place and interact with products. By far, clothing stores have benefitted the most from these options as customers can virtually try on fashions, with accurate fit and complex features.

How Will Mixed Reality Impact Life Post-Pandemic?

While we are shifting to a new digital era, we are still in the early days of this transformation. According to Deloitte Insights, there will be many tipping points in augmented shopping and entertainment as it becomes the new norm. Mixed reality will allow shoppers to select a product type and interactively find the perfect dimension, color or style while also trying or testing them out in the comfort of their own homes. In other instances, a user can hold a conversation with a celebrity or hop into a concert at any time or any place.

Over 1 billion consumers currently have smartphones or tablets that can deliver augmented experiences. Within the next year, 100 million consumers are expected to shop using AR. It is safe to say that the appetite for a mixed medium is increasing.

Further, there are many exciting projects in the works to fully integrate virtual, augmented and mixed reality into our lives. A Vermont-based startup OVR Technology is researching the Architecture of Scent to create a fully immersive experience. Olfactory Virtual Reality’s (OVR’s) hardware, called ION, consists of a novel piezoelectric atomization technology that outputs scent molecules.

What do you think the future of mixed reality will look like?