You’re on a beautiful beach, sand stretching to your sides, waves of warm water ahead, and a refreshing mojito in your hand. You try to take a sip and nothing happens.
You’re standing on the moon. The view is mostly black and empty, though there are some pretty nice stars out there and that pale blue dot is kind of neat. You try to step towards it but you’re frozen in place.
You’re in the ring with Muhammad Ali. His right hook is coming in fast and you try to block but you can’t seem to get your hands up or lean out of the way and—K.O.
Virtual reality without motion tracking is like a movie without sound. It can still be great, but there’s something fundamental missing. After all, we can interact with physical reality—we should be able to interact with virtual reality just the same. Today’s VR systems often include handheld controllers for hand tracking or sensors for full body tracking, but there’s still a long ways to go before VR becomes the holodeck experience we’ve all dreamed of.
In this article, we’ll take a look at some of the latest news from the world of VR tracking.
Forget Handheld Controllers—Let’s Use Magnets
At the recent Regeneron International Science and Engineering Fair (ISEF), hosted virtually for the first time ever on ProjectBoard (owned by engineering.com), high schooler Yashua Evans won an ISEF 2021 Special Award for his project, VR But Better.
Evans’ project comprised two ideas, both aimed at improving VR immersion through better motion sensors. The first idea was the Geomagnetic Tracker, a new approach to VR hand tracking that uses two magnetometers to sense the rotation of a user’s elbow and shoulder and translate it into hand movement. This would enable hand-free hand tracking, according to Evans, and solve problems like drift and occlusion that can disrupt other motion trackers.
Evans paired the Geomagnetic Tracker with a previous project of his, the Electromyographic Controller, which reads muscle activity through electrodes placed on the user’s forearm. These signals are translated into hand controls, such as opening and closing the hand as an alternative to pressing a button on a controller.
Evans’ second idea for better VR was the Pressure Distribution Translator, a plastic mat with pressure sensors to determine the direction a user is leaning in VR.
The prototype for Evans’ Electromyographic and Geomagnetic Controller worked as expected, though the geomagnetic sensors had an undesirable delay that Evans believes can be addressed with a dedicated magnetometer chipset. His Pressure Distribution Translator did not provide accurate readings, he admitted, and he plans to use a faster microcontroller for future iterations.
Evans is a natural advocate for science, and his ISEF 2021 Special Award is well deserved. “Science is one of the biggest inspirations for me because it gives normal people the ability to do unimaginable things,” he wrote.
Using Webcams for Full Body Tracking
A Japanese group called Akiya Research Institute VRlab has teased what could be an affordable approach to motion tracking for VR or motion capture applications. Last month on Twitter, Akiya announced forthcoming software called MocapForAll that can be used for full body VR tracking using at least two webcams or other cameras. The software is scheduled to be released this month.
Though this isn’t the first camera-based motion tracker ever to hit the market—Microsoft’s discontinued Kinect remains popular with some VR users—MocapForAll could potentially offer an affordable software solution that allows users to leverage their current hardware. We’ll be sure to track this story as it develops.
They’re Watching You, Private Eyes
Eye tracking is an increasingly popular strategy in virtual reality, since the focus of a user’s attention is valuable data for just about any application. Head-mounted displays such as the HTC Vive Pro Eye offer eye tracking as a headline feature. NVIDIA RTX graphics cards include a feature called foveated rendering for VR apps that will focus resources on the part of a scene that users are actually looking at. It seems likely that eye tracking will be a common part of the VR landscape moving forward.
VR company WorldViz is now attempting to make it easier for users to deploy VR eye tracking experiments with the release of software called SightLab VR. According to WorldViz, SightLab VR is a drag and drop tool to quickly develop VR scenes and collect eye tracking data. Researchers can use the software to measure eye gaze, pupil size and fixation locations, and scan paths throughout a VR scene. A retail store could set up a virtual storefront and learn the locations of their blind spots, for example, or a museum could set up a virtual exhibit and see what artwork was most popular.
“One of the challenges with performing research using virtual reality and eye tracking is the complexity of building the initial experiment. With built-in tools specifically for setting up eye tracking experiments in VR, SightLab VR got us up and running and doing our research in very little time with minimal effort,” praised Joy Lee of Maastricht University.
Hate The Treadmill? VR Will Fix That
Virtuix is a VR company known for developing the Omni, an bowl-shaped omnidirectional treadmill made to simulate real movement in VR applications. To move in VR, users can walk or run in any direction on the Omni, never moving in this world but covering miles in another. The Omni is a commercial system found in arcades and other entertainment venues.
Now, Virtuix has announced that its raised over $19 million for the Omni One, a consumer version of the Omni. “With Omni One, your home becomes a portal into new worlds and gaming adventures like never before,” said Jan Goetgeluk, Virtuix's founder and CEO. “For the first time, you're no longer restricted by the limited space in your home. You can roam endlessly around immersive virtual worlds as you would in real life, using your whole body.”
Omni One is not just the treadmill; it also includes a rotating arm connected to a special vest worn by the user to capture the full range of user motion. At a projected price of $1995, the Omni One will ship with its own standalone VR headset and game store. It’s not clear if the headset is developed by Virtuix or if other headsets will be compatible with the system.
Virtuix is pitching the Omni One as an alternative to home exercise systems like the wildly popular Peloton, and is explicitly copying that company’s subscription-based business model in its pitch to investors.
“Omni One isn't just a next-level gaming device, it also keeps you in shape by burning calories while gaming. Think of Omni One as a Peloton bike for gamers, and for parents who want to get their kids off the couch!” Goetgeluk said.