Google’s DeepDream software is a learning AI application with the purpose of advancing recognition software for faces, animals and other patterns. What’s particularly interesting about it is its unexpected surreal results, which has sparked significant online chatter.
In an experiment, software engineers from Google’s research labs ran various pictures through a neural network. The program was asked to identify specific patterns in images ranging from scenery, to objects, to random noise. The program then alters the image to match its expectations given user parameters and the patterns it identified. The process has produced some psychedelic and often nightmarish images.
According to Google’s Research blog, their Artificial Neural Networks are designed to make progress in image classification and speech recognition software. The software engineers “trained” the neural network feeding it millions of example images. To make the network focus on finding a particular pattern, users adjust the parameters until it gives the desired classifications.
Each neural network typically contains 10 to 30 stacked layers of artificial neurons. Users feed their submitted image through the input layer and each layer talks with the next sequentially. When the image finally arrives through the output layer, the resulting image should fit the user’s desired parameters.
“We know that after training, each layer progressively extracts higher and higher level features of the image, until the final layer essentially makes a decision on what the image shows,” reads the blog entry by software engineers Alexander Mordvinstev, Mike Tyka, and intern Christopher Olah. “For example, the first layer maybe looks for edges and corners. Intermediate layers interpret the basic features to look for overall shapes or components, like a door or a leaf. The final few layers assemble those into complete interpretations – these [artificial] neurons activate in response to very complex things such as entire buildings or trees.”
The team describes a scenario where a user wanted to find bananas in a given image of random noise. The neural network will “…gradually tweak the image towards what [it] considers [is] a banana. By itself, that doesn’t work very well, but it does if we impose a prior constraint that the image should have similar statistics to natural images, such as neighbouring pixels needing to be correlated.”
The software can be trained to associate objects with other objects such as a dumbbell with an arm. However, this can backfire when an image being processed doesn’t include an arm. This tells the software engineers they must teach the network to recognize that not all images with dumbbells require an arm.
This error in the training can also create a feedback loop. When a network trained to find dogs sees a cloud that kind of looks like a dog it will tweak the image until it contains a dog.
In essence, neural networks trained to discriminate between different kinds of images retained enough information to generate images as well, resulting in the #deepdream phenomenon sweeping twitter.
Make your own neural network inspired images with an open source visualization tool #deepdream http://t.co/oIxBHZ7Q5G pic.twitter.com/d0IG43BMCa
— Google Research (@googleresearch) July 1, 2015
We ran the #Chronophage through Google's #deepdream algorithm. It became less surreal. pic.twitter.com/tJwN8SNyls
— Cambridge University (@Cambridge_Uni) July 3, 2015
results on video are promising, stable low level features, high level features are too sensitive to noise #deepdream pic.twitter.com/gScLe5Pk0U
— Kyle McDonald (@kcimc) July 2, 2015
Networks trained to find particular patterns can be left to over-analyze images, finding patterns within patterns. Google’s software engineers call this process “Inceptionism,” in reference to the
neural net architecture. When used, Inceptionism has resulted in imagery that harkens to real-world biological designs and fractal-like patterns in the background of images.
ecause of bias in the network, certain images seem to attract certain features. Horizon lines are usually filled in with towers and pagodas, trees develop images of buildings, while birds and insects appear in leaves.
Inceptionism, despite how random and bizarre it seems, has its purpose: “This technique gives us a qualitative sense of the level of abstraction that a particular layer has achieved in its understanding of images,” the blog reads. “The techniques presented here help us understand and visualize how neural networks are able to carry out difficult classification tasks, improve network architecture and check what the network has learned during training. It also makes us wonder whether neural networks could become a tool for artists—a new way to remix visual concepts—or perhaps even shed a little light on the roots of the creative process in general.”
Google has made the code for the tool public, posting it on open source site Github, allowing anyone to download it and try it for themselves. Let us know your thoughts in the comments below.