Playing Video Games with Direct Brain Stimulation

For the first time, researchers have demonstrated the feasibility of direct brain stimulation as an alternative to sensory input. The research team designed an experiment wherein test subjects had to play a simple 2D video game, with a catch: the players couldn’t see the game directly. Instead, they relied only on the direct stimulation of their brains via a technique called transcranial magnetic stimulation (TMS) to play the game successfully.

An example of one of the mazes the players had to traverse. (Image courtesy of University of Washington.)

Brain in a Maze

The game is simple: players must navigate from one end of a maze to the other. It’s made even easier by the fact that there are only two types of turns, forward and down. This design allows for binary encoding of information about the player’s position, which the researchers accomplished with TMS.

TMS is a common neurological technique that uses a magnetic coil, placed near the head, to induce small electric currents in the brain. By placing the coil over the primary visual cortex, it’s possible to produce a temporary visual perception called a phosphene.

For the game, the TMS pulse intensity was dependent on the position of the player avatar. Being directly in front of a wall caused players to perceive a phosphene, while being far from a wall did not. Interpreting this stimulation correctly would therefore allow players to reach the end of the maze. The subjects navigated 21 different mazes in this fashion, with 7 control mazes that lacked TMS stimulation.


One of the test subjects navigating a maze, unseen by him, yet visible on the screen in the background. The TMS coil on the back of his head provides information about the maze by stimulating the perception of phosphenes. See full paper for details. (Image courtesy of University of Washington.)

A Sixth Sense

“We’re essentially trying to give humans a sixth sense,” said lead researcher Darby Losey. “So much effort in this field of neural engineering has focused on decoding information from the brain. We’re interested in how you can encode information into the brain.”

The results of the experiment provide encouraging insight into this problem. The five test subjects successfully navigated an average of 92 percent of the steps in the virtual mazes by way of direct brain stimulation, compared to only 15 percent in the control cases. Furthermore, the subjects improved over time, suggesting they were actively learning to better interpret the brain stimulation.

“The fundamental question we wanted to answer was: Can the brain make use of artificial information that it’s never seen before, that is delivered directly to the brain, to navigate a virtual world or do useful tasks without other sensory input?” asks researcher Rajesh Rao. “And the answer is yes.”


Finally, A Use for Brains

Gamers will likely be envisioning the potential of direct brain stimulation for virtual reality, and the possibilities of new methods for interacting with games. But the research also suggests more immediate applications, even in the simple binary form of the experiment. For example, a similar method of phosphene stimulation could prove useful to blind people navigating unfamiliar environments. They’ll have to wait a bit longer, however.

“The technology is not there yet — the tool we use to stimulate the brain is a bulky piece of equipment that you wouldn’t carry around with you,” said researcher Andrea Stoccot. “But eventually we might be able to replace the hardware with something that’s amenable to real world applications.”

Members of the research team have co-founded a startup, Neubay, aimed at commercializing their ideas. “We look at this as a very small step toward the grander vision of providing rich sensory input to the brain directly and noninvasively,” said Rao. “Over the long term, this could have profound implications for assisting people with sensory deficits while also paving the way for more realistic virtual reality experiences.”

There’s still a lot to learn about the brain to make significant progress in this area. For more about how we might do this, read Understanding the Brain with Super-Small Needles.