Unity and DeepMind: Where Does AI Come into the Physical World?

The Gaming Connection

Unity has joined forces with DeepMind, the artificial intelligence company acquired by Alphabet (Google) in 2014.

If you didn’t know, Unity is probably the most popular development platform for creating 3D content in the world. Developers have used it to create half of all mobile games and a majority of augmented reality and virtual reality content consumed today.

Unity opened its doors in 2005 as a platform to enable gaming developers of different experience levels the ability to create games. Its general-purpose engine grew ideologically from the requirements that it be flexible yet robust enough to support different platforms and types of games.

Unity is used for developing mobile games and 3D content for AR and VR. It is also used to create dynamic simulation programs with assets like the aircraft pictured above. (Image courtesy of Unity Flight Simulator).

The company that makes up the other half of the agreement is Alphabet’s DeepMind. You might remember DeepMind from its infamous triumph in 2016, when its AlphaGo rig defeated Go world champion Lee Sedol in a series match—four games to five. Go was widely considered to be too intuitive of a game too high a level of variability for any computer program to win in the manner that IBM’s Deep Blue did in 1996—when that computer program won game one of a six-game match against Gary Kasparov.

Prior to the DeepMind partnership announcement, Unity gave their community the ability to download Unity Machine Learning (ML) Agents, a software built to train ML agents with reinforcement learning and evolutionary methods via a simple Python API. The team at Unity isn’t new to AI but they are certainly not as sophisticated in their practice as DeepMind. According to Unity, these ML agents enable academic researchers “to study complex behaviors with realistic physics baked in to simulations and allows industrial and enterprise researchers to implement large-scale parallel training regimes for robotics, autonomous vehicles, and other industrial applications.” They held a contest (which ended in February 2018) for developers to create interesting Unity games featuring their ML Agents. The video above is the third-place winner, called Hide/Escape, submitted by David Bausch from Austin, Texas. (Video courtesy of YouTube.)

Conceived and developed in 2010, DeepMind’s artificial intelligence technology grew from the human minds and actions of University College of London students Demis Hassabis and Shane Legg, who taught early versions of what would become DeepMind it to learn by playing simple Atari 2600 video games.

At the Gatsby Computational Neuroscience Unit on campus, Hassabis and Legg introduced their AI to what are now considered to be videogame classics like Breakout, Pong and Space Invaders—one game at a time—without any pre-existing knowledge of each game’s rules. (Video courtesy of YouTube.)

What the Partnership Between Unity and DeepMind Really Means

The partnership is an attempt to close what’s known in AI and robotics as the reality gap. The reality gap is a series of gaps between simulation and testing as well as between digital AI and robotic AI. In order to mimic mammalian (human) intelligence, which was built in the physical world, DeepMind and Unity are hoping to create virtual environments built from Unity’s Machine Learning Agents platform and DeepMind’s algorithmic research.

The goal is to collect data from simulations that are more realistic than from those of which DeepMind was initially trained. Could you train a digital avatar in a simulator so indistinct from a realistic environment that a robot could upload useful data solutions from the millions of simulated trials? Could one derive meaningful information from the digital version of a robot. Could data derived from the digital world share enough attributes containing characteristics that are compatible and transferrable to a robot and its real physical environment?

After all, if you could train an AI agent in a simulation of a physical location in a realistic enough way, and the data collected could be transferred to a robot with similar or exactly the same properties, then you have could program the robot with a mastery of its task within a given environment. It’s incentive to act could be designated for a digital replica of a physical environment, then carry out instructions from that data in the actual physical environment.

DeepMind produced this groundbreaking defeat of a 9th dan world champion by developing an innovative computing technique called deep reinforcement learning, which combines two other forms of narrow AI—deep learning and reinforcement learning. Due to Go’s intuitive nature, many thought such a feat was more than a decade away from becoming a new triumph of computing over humanity’s best and brightest. (Image courtesy of DeepMind.)

Since DeepMind became a semi-independent subsidiary of Google, who acquired it in 2014 for USD 500 million, the acquisition has yielded improvements in several areas critical to Google’s products and services. For example, Google has deployed DeepMind’s algorithms to improve its search engine and the cooling systems of its data centers. Another example is a collaboration between Google engineers and DeepMind researchers, which ultimately helped Google Play make personalized app recommendations. Yet another internal cross-pollination yielded some interesting machine-learning features for the ninth version of Android Pie (the Android operating system), including helping Android computing devices maximize their energy efficiency and streamline their ease-of-use for Google’s hardware user base.

With DeepMind, Unity developers are planning on creating massive virtual environments for researchers to study autonomous AI agents and develop new types of algorithms. These new algorithms will be relevant for different situations that advanced technology will be facing in the physical world.

It's easy to imagine how advanced technology like robotics, drones and autonomous cars could then apply these algorithms in actions taken in the physical world. Perhaps autonomous cars will be able to navigate better when unexpected situations like accidents or construction affect traffic conditions. Perhaps swarm drones will be able to coordinate and navigate as a collective more efficiently and in a greater number of more diverse patterns. For robotics engineers, new algorithms may offer robots a new level of autonomy, and perhaps a better way to incorporate sensor feedback data into relevant data to learn how to adapt and overcome physical obstacles and various limitations.

Using Unity for Real-World Simulations

Gathering real-world data has its limits, and it certainly has its costs. When gathering real-world data such as logging autonomous driving miles for testing becomes prohibitive for cost reasons or very practical reasons (like not risking injury or death by autonomous car) adding synthetic data gathered via Unity’s photorealistic simulation capabilities is proving useful to researchers from companies and academia.

The main goal of autonomous driving is to optimize traffic conditions and coordinate collectively to control flows so that accidents and injuries become far rarer than they are right now. Unity partnered with the City of Bellevue in Washington State for Vision Zero, the name of a project designed to reduce fatalities to zero.

For example, a project known as SYNTHIA, developed solely on Unity by the Computer Vision Center (CVC) at Universitat Autonoma de Barcelona, is an attempt by part of an academic institution to create a system of computer vision that can identify unsafe intersections by examining intersections using a network of cameras to gather data which are filtered through a set of criteria. It recognizes objects in video streams and gathers data about the frequency and character of traffic incidents, then passes that information to traffic planners for review. Unity has an ability to synthesize photorealistic data based on the real world and this helps the computer vision system perform better at object recognition.

Using Unity to Control Robots

Human-Robot Interaction is like many fields today, a multi-disciplinary field which needs computer scientists, engineers and experts on human behaviors to create realistic animations for humanoid and non-humanoid robots. The Unity Game 3D Game Engine has been used to synch up robots with its modeling environments, working like a digital joystick with its robot counterpart. Researchers at the University of Canterbury in New Zealand created a program called The Robot Engine used Unity’s simple design principle of empowering non-programmers to create dynamic games, harnessing it to control a few different Arduino-based robot platforms. In a paper published in 2015, researchers responsible for creating The Robot Engine explained how easy it would be to extend the application to other robotics platforms.

Bottom Line

Using 3D scanned real-world environments to train artificially intelligent robots to learn how to better navigate real-world obstacles in photorealistic simulations and controlling them by wielding their digital twin in Unity until they become autonomous is strange.

Theoretically, in the future, if a simulator could create a digital replica of a physical environment in Unity 3D, someone could set up the following experiment: In Unity 3D, an AI agent in a digital avatar repeatedly navigates a simulated virtual environment that is a digital replica of a real physical environment using variations of DeepMind’s reinforcement learning. If the parameters of this digital- AI agent-avatar’s motion were limited to the sets of ranges of motion in a real robotic-AI-avatar counterpart, the reinforcement learning accomplished by the digital AI avatar in simulation of this reality capture environment could be modulated in software to build a series of algorithms that would help the real robotic counterpart navigate the physical environment with data that was scanned and transformed into the Unity simulation.

Currently, machine-learning is only being used by robotics in Open AI’s Dactyl experiment, which uses reinforcement learning to “learn dexterous in-hand manipulation policies which can perform vision-based object reorientation on a physical Shadow Dexterous Hand.” (Image courtesy of Open AI.)

Don’t worry about killer robotic AI just yet, after all Google wanted to combine DeepMind with Boston Dynamics (DeepMind declined, and this may be part of the reason why Google sold it to SoftBank) but that didn’t happen. There are a lot of interoperability and complexity issues with transferring machine learning from simulations into robotics, and the amount of hardware necessary to facilitate this bridging of learned data sets is very cost-prohibitive at the moment. Making simulations both more realistic for machine learning is just part of the way researchers, engineers and scientists are looking to close the reality gap between digital agents and their robotic counterparts a bit further.

Who knows what the future holds?