Helsinki-based XR hardware and software developer Varjo has embarked on a lofty goal: to redefine reality. The company is currently pushing the needle on virtual reality (VR) capabilities with its Varjo Reality Cloud release. The platform enables users to 3D scan their surroundings using the Varjo XR-3 headset and “teleport” another person into that environment for a high-fidelity experience. In this way, the physical world can morph into a virtual world shared via the cloud.
It’s a real-time way to virtually share the experience of one’s surroundings in a manner that could transform the world of communication by facilitating universal collaboration in a burgeoning metaverse—also called the omniverse or mirrorworld.
“We believe that Varjo’s vision for the metaverse will elevate humanity during the next decade more than any other technology in the world,” said Timo Toikkanen, CEO of Varjo. “What we’re building with our vision for the Varjo Reality Cloud will release our physical reality from the laws of physics. The programmable world that once existed only behind our screens can now merge with our surrounding reality—forever changing the choreography of everyday life.”
The Varjo Reality Cloud has been five years in the making, requiring advances in foundational technologies such as human-eye resolution, low-latency video passthrough, integrated eye tracking and headset LiDAR. The Varjo Reality Cloud uses a proprietary foveated transport algorithm that allows users to stream real-time human-eye resolution, wide field of view 3D video feeds to other devices in single megabytes per second. This enables the sharing and editing of one’s environment for collaboration with others without time and space constraints.
In addition to developing the virtual teleportation system, Varjo has also acquired Dimension10, a Norwegian software company that pioneers industrial 3D collaboration. The company has a virtual meeting suite designed for architecture, engineering and construction applications that will be used to facilitate virtual collaboration in the Varjo Reality Cloud.
When Varjo began its development of the Varjo Reality Cloud in 2016, it first focused on developing a VR system that would produce the world as clearly as seen by the human eye. This was done through its Bionic display, a high, human-eye resolution screen delivered across a 115-degree view field. The Varjo XR-3 headset offers a high-fidelity mixed reality capability that blends the real with the virtual through a 12-megapixel, low-latency video passthrough technology. By using LiDAR and stereo RGB, Varjo claims a pixel-perfect depth awareness. The headset also delivers natural interactions through 200Hz integrated eye tracking v5 Ultraleap hand tracking.
“We wouldn’t be here without the visionaries who use our products,” said Annaleena Kuronen, Varjo’s head of communications. “Our users have pushed us beyond what we originally thought was possible. During the last four years, we’ve transformed how people in some of the most demanding industries in the world are trained, such as pilots, nuclear power plant operators, astronauts and surgeons. We’ve changed the way designers can collaborate across continents and how researchers conduct studies in immersive environments.”
According to Kuronen, Varjo’s VR products were used by astronauts-in-training who virtually performed a crew space mission from prelaunch to take off by docking at the International Space Station and landing back on Earth—all in VR.
Although VR teleportation could be used extensively in gaming, Varjo is marketing its products for professional and industrial applications. However, the uses of teleportation VR are manifold. For example, the COVID-19 pandemic has deepened the desire for virtual connections globally. In addition, the rise of remote working has led to concerns among business managers that teams could lose a sense of cohesion, or that relying on teleconferencing could hamper collaborative creativity for companies. However, as VR teleportation advances, these concerns could be assuaged as teams could collaborate within the same virtual space. Furthermore, the technology would help maintain human connection in situations like the pandemic when family members and friends are separated in the physical sphere.
The next step in Varjo’s VR development journey is to harness the cloud’s extensive capabilities to share the entire physical reality in a virtual real-time setting through the Varjo Reality Cloud. The idea is not just to have a virtual space that’s merely photorealistic down to the last pixel but also to allow users to edit and modify their surroundings, such as changing the lighting in a room or adding objects.
Varjo isn’t the only company in the VR teleportation development business. Microsoft developed a similar holoportation system, which uses 3D capture technology to create an image of a person inserted into a virtual space. For example, during a Microsoft Ignite conference in March, remote attendees inhabited the conference as avatars in a holographically constructed world. The technology is enabled by the HoloLens 2 headset and Microsoft Mesh, a mixed reality platform by Azure that allows people in different places to share a holographic experience using other devices.
“This has been the dream for mixed reality, the idea from the very beginning,” said Microsoft Technical Fellow Alex Kipman. “You can actually feel like you’re in the same place with someone sharing content, or you can teleport from different mixed reality devices and be present with people even when you’re not physically together.”
During the conference, Kipman appeared onstage as a virtual holoportation of his physical body and was joined by filmmaker and ocean explorer James Cameron and CEO of AR company Niantic.
OceanX, a nonprofit that fosters education and awareness about oceans, is also collaborating with Microsoft to create a Mesh-enabled holographic laboratory on a ship where scientists can gather virtually to see 3D holograms of ocean scenery and exploring vessels.
“The idea is to take all this amazing scientific data we’re collecting and bring it into a holographic setting and use it as a way to guide scientific missions in real time,” said Vincent Pieribone, vice chairman of OceanX.
Researchers with the Microsoft HoloLens 2 would use Mesh to sit as an avatar at a table with colleagues from around the world and interface with a holographic image of ocean topography or wildlife and discuss topics in real time.
Like Varjo’s products, the HoloLens 2 is aimed at professional users in manufacturing, health care, education and research. For example, Microsoft states that the device yields a 90 percent increase in efficiency for assembly during manufacturing, a 40 percent reduction in travel time for auto technicians who can use the HoloLens 2 to troubleshoot issues remotely, and a 30 percent decrease in the amount of time health care professionals spent doing rounds during the COVID-19 pandemic.
In addition to facilitating space flight training, Varjo claims that its headset results in a 90 percent reduction in the cost of training heavy machine operators. It has also piloted a VR training for control room operators of a nuclear power plant in Finland.
The HoloLens 2 and Varjo’s XR-3 headset are both currently available on the market. Varjo’s Reality Cloud is not yet ready for consumers; however, Alpha Access partners will have an opportunity to use the technology later this year.