A.I. Uses Deep Learning to Beat Humans at DOOM

An artificial intelligence agent has proven to be the ultimate survivor in the classic video game Doom—outplaying both the game's built-in AI agents and human players.

Created by two computer science students at Carnegie Mellon University, Devendra Chaplot and Guillaume Lample, the AI used deep-learning techniques to negotiate the game's 3-D environment. This is still challenging after more than two decades because players must act based only on the portion of the game visible on the screen.

This work follows the in the footsteps of Google's DeepMind, which used deep-learning methods to master two-dimensional Atari 2600 videogames, and earlier this year defeated a world-class professional player in the board game Go. In contrast to the limited information provided in Doom, both Atari and Go give players a view of the entire playing field.

“The fact that their bot could actually compete with average human beings is impressive,” said Ruslan Salakhutdinov, an associate professor of machine learning who was not involved in the student project. “Simply navigating a 3-D world, much less competing successfully in this game environment, is a challenge for such AI agents.”

Chaplot and Lample began getting online attention for their work after posting a research paper, which they have submitted for review to a leading AI conference. In the Visual Doom AI Competition—in which AI agents play against each other in deathmatches—the duo's agent placed second to a team from Facebook in one track and second to a team from Intel in the competition's other track.

Teaching an AI to Play Doom

Chaplot explained that humans have natural advantages in chasing and dodging enemies in Doom's 3-D world. The game's own built-in agents have to “cheat” by accessing maps and other game information in order to be competitive. He and Lample trained their AI agent, Arnold, to play the game based only on what is visible on the screen, just like human players.

To do so, they combined several existing deep learning techniques based on neural networks in their own unique architecture. When the player is navigating through the game, it employs a Deep Q-Network, a reinforcement learning architecture that DeepMind used to master Atari games. When an enemy is in sight, the agent switches to a Deep Recurrent Q-Network, which includes a long short-term memory (LSTM) module that helps the agent track the enemy's movements and predict where to shoot.

Though the AI agent relies on only visual information to play the game, Chaplot and Lample used an application program interface (API) to access the game engine during training. This helped the agent learn how to identify enemies and game pieces more quickly, Chaplot explained. Without this aid, they found the agent learned almost nothing in 50 hours of simulated game play, equivalent to more than 500 hours of computer time.

Not only is Arnold fast and an accurate shot, but it has also learned to dodge shots, making it hard to eliminate.

"We didn't train anything to kill humans," said Chaplot. "We just trained it to play a game." That being said, Chaplot added that the deep reinforcement learning techniques they used to teach their AI agent could someday help self-driving cars operate safely on real-world streets or to train robots to do a wide variety of tasks to help people.

Interested in video games? Check out our Top 10 Video Games for Engineers.