Using Soft Robotics in Virtual Reality

Engineers are using soft robotics technology to make light, flexible gloves that allow users to feel tactile feedback when they interact with virtual reality environments. The researchers used the gloves to realistically simulate the tactile feeling of playing a virtual piano keyboard.

The engineers recently presented their research, which is still at the prototype stage, at the Society for Imaging Science and Technology conference, “Engineering Reality of Virtual Reality” in Burlingame, Calif.

Currently, VR user interfaces consist of remote-like devices that vibrate when a user touches a virtual surface or object. "They're not realistic," said Jurgen Schulze, a researcher at the Qualcomm Institute at UC San Diego and one of the paper's senior authors.

"You can't touch anything, or feel resistance when you're pushing a button. By contrast, we are trying to make the user feel like they're in the actual environment from a tactile point of view."

A glove powered by soft robotics is allowing these Ph.D. students to play piano in VR. (Image courtesy of Jacobs School of Engineering/UC San Diego.)
Other research teams and industry have worked on gloves as VR interfaces. But these are bulky and made from heavy materials. The glove the engineers developed has a soft exoskeleton equipped with soft robotic muscles that make it much lighter and easier to use.

"This is a first prototype but it is surprisingly effective," said Michael Tolley, a mechanical engineering professor at the Jacobs School of Engineering at UC San Diego and also a senior author.

The system involves three main components:

  • A Leap Motion sensor that detects the position and movement of the user's hands
  • A custom fluidic control board that controls the gloves' movements
  • A soft robotic components in the glove that individually inflate or deflate to mimic the forces that the user would encounter in the VR environment

The system interacts with a computer that displays a virtual piano keyboard with a river and trees in the background.

One key element in the gloves' design is a type of soft robotic component called a McKibben muscle, essentially latex chambers covered with braided fibers. The muscles respond like springs to apply force when the user moves their fingers. The board controls the muscles by inflating and deflating them.

The researchers 3D-printed a mold to make the gloves' soft exoskeleton. This will make the devices easier to manufacture and suitable for mass production, they said. They used silicone rubber for the exoskeleton, with Velcro straps embedded at the joints.

Soft robotic muscles are a key component of the gloves. (Image courtesy of Jacobs School of Engineering/UC San Diego.)
Engineers conducted an informal pilot study of 15 users, including two VR interface experts. All tried the demo which allowed them to play the piano in VR. They all agreed that the gloves increased the immersive experience. They described it as "mesmerizing" and "amazing."

The engineers are working on making the glove cheaper, less bulky and more portable. They also would like to bypass the Leap Motion device altogether to make system more compact.

"Our final goal is to create a device that provides a richer experience in VR," Tolley said. "But you could imagine it being used for surgery and video games, among other applications."

Source: University of California – San Diego