Last Thursday, Google’s DeepMind made the cover of Science with a paper about AlphaCode, a deep learning model for writing computer code. Using transformer-based language models, AlphaCode can write computer programs at a level consistent with an average human programmer with about a year of training, according to DeepMind.
AlphaCode competed on the Codeforces platform, in which tens of thousands of programmers put their coding skills to the test. DeepMind deployed AlphaCode in ten competitions. The AI code generator placed at the level of the median competitor, ranking in the top 54.3 percent of competitors in the simulated challenges. DeepMind describes this as the first time AI code generation has been competitive in a programming competition.
"I can safely say the results of AlphaCode exceeded my expectations,” said Mike Mirzayanov, founder of Codeforces. “I was skeptical because even in simple competitive problems it is often required not only to implement the algorithm, but also (and this is the most difficult part) to invent it. AlphaCode managed to perform at the level of a promising new competitor. I can't wait to see what lies ahead.”
Although there is still further development needed for AI-generated code to reach its full potential, DeepMind is invested in the future of these solutions. AlphaCode could shift how programmers approach their work. In the not-too-distant future, humans may formulate problems that ultimately get solved by AI.
AlphaCode is not the first AI-powered solution aimed at creating code. GitHub also has a popular tool called Copilot that uses AI to generate code. However, Copilot has not demonstrated a similar level of competency when challenged directly by human competitors. Copilot is powered by OpenAI’s Codex, which also powers Microsoft’s GPT-3. DeepMind describes AlphaCode as distinct from these solutions, as competitive coding problems usually require some degree of interpretation and critical thinking to solve—a different task from simply finishing existing code.