Researchers Are on a Quest to Combine Artificial Intelligence and Supercomputers

Despite recent claims from Google about achieving quantum supremacy, the world’s fastest and most reliable computers by a long shot are supercomputers, also known as high performance computers (HPC). The architecture and hardware inside today’s fastest supercomputers is mind-boggling. Interestingly, researchers and computer scientists who use supercomputers have only recently begun paring supercomputers with artificial intelligence (AI) applications.

Back in January 2019, the IBM Summit (the world’s most powerful computer) ran a complicated application based on Google’s open-source TensorFlow. In short, the computer ran a complicated machine learning model of climate research. But AI researchers struggle to access supercomputers for a couple of reasons. Due to high demand, access is limited. Complicating matters is the fact that preparing applications to run smoothly on supercomputers is time intensive and complex.

What Are Supercomputers and How Are They Used?

Supercomputers are basically similar to workstations, except that they are stacked to the gills with CPUs, GPUs, RAM and very clever engineering. The software is incredible as well, but most supercomputers run Linux to perform endless calculations at blistering speed. Using parallel processing, supercomputers divide and distribute computing tasks to thousands of CPUs simultaneously. As a result, they are the lifeblood of sophisticated research and analysis. The world’s fastest supercomputers are used in academic and government-funded research facilities, government agencies and multinational corporations to perform a variety of computing-intensive tasks.

The National Oceanic and Atmospheric Administration (NOAA) uses supercomputers to collate data aggregated from local and global sources. From this massive swath of information, statistical and mathematical database operations are performed to help climatologists and meteorologists predict various local and global weather events.

The National Security Agency (NSA) in the United States and analogous global government agencies use supercomputers to monitor all digital communication transactions around the globe. Hostile governments and terrorist entities use sophisticated encryption (which is always more advanced than decryption) to communicate via satellite, radio, the internet and mobile phone transmissions. Supercomputers are used to detect threat patterns that can be monitored and assessed.

Corporations that use data science as part of their core products and services use supercomputers to keep business booming. For example, life insurance companies use these computers to reduce actuarial risks to the lowest possible levels. Brokerage firms use supercomputers to help analyze market trends and execute stock trades with lightning speed. Retail giants like Amazon and Walmart use them to spot market trends and sales patterns to make logistical adjustments along their supply chain as needed.

Astronomers at NASA use supercomputers to analyze data streams from radio telescopes, exploratory space probes, ground-based optical telescopes and satellites in orbit for a variety of research-related purposes. Physicists at the European Organization for Nuclear Research (CERN) used supercomputers to find the Higgs boson particle through supercomputer-powered analyses of the massive troves of data created by the Large Hadron Collider.

The World’s Fastest Supercomputers Are Becoming More AI Friendly

Supercomputers are ranked by the High-Performance LINPAC (HPL) Benchmark for distributed memory computers. TOP500 provides a periodic list of the top-ranked supercomputers in the world.

This is the world’s most powerful supercomputer, the IBM-built Summit system, which is used by residents of Oak Ridge National Laboratory and scored an HPL result of 148.6 petaflops. (Image courtesy of Oak Ridge National Laboratory.)

With an HPL result of 95.6 petaflops, the IBM-built Sierra system at Lawrence Livermore Laboratory is number two supercomputer in the world. (Image courtesy of Lawrence Livermore National Lab.)

At number three with an HPL score of 93.0 petaflops is the Sunway TaihuLight supercomputer. It was designed and constructed by China’s National Research Center of Parallel Computer Engineering & Technology (NRCPC) and runs only on Sunway’s SW26010 processors. (Image courtesy of NRCPC.)

Located at the National Supercomputer Center in China, the Tianhe-2A (Milky Way-2A) is ranked at number four among the world’s fastest supercomputers. It scored 61.4 petaflops on the HPL benchmark and runs on Matrix-2000 accelerators and Intel Xeon processors. (Image courtesy of the National Supercomputer Center.)

Gaze your eyes upon the Frontera, the fifth fastest supercomputer in the world. It’s a Dell C6420 system that scored an HPL of 23.5 petaflops. It’s powered by Xeon Platinum processors and was deployed at the Texas Advanced Computing Center at the University of Texas at Austin in 2018. (Image courtesy of the University of Texas.)

The supercomputer picture above is a Cray XC50 supercomputer at the Swiss National Supercomputing Centre (CSCS). It is the most powerful supercomputer in Europe (sixth most powerful in the world) and received a 21.2 petaflops mark on the HPL. Cray recently showed new software that lessens the difficulty of running programs that utilize geospatial data like satellite imagery. Cray also has a deep-learning plug-in to increase the efficiency of training models used for a variety of purposes. (Image courtesy of the Swiss National Supercomputing Centre.)

This Cray XC40 resides at Sandia National Laboratories, Los Alamos National Laboratory, and is ranked seventh in the world. Named Trinity, it scored 20.2 petaflops on the HPL. (Image courtesy of Los Alamos National Laboratory.)

The eighth-fastest supercomputer in the world is called AI Bridging Cloud Infrastructure (ABCI) and resides at Japan’s National Institute of Advanced Industrial Science and Technology (AIST). This supercomputer is becoming easier for AI researchers to use thanks to “software containers,” which were developed by NVIDIA. The software containers bundle applications with digital libraries and the analogous software needed to run AI applications like deep-learning models. It received a mark of 19.9 petaflops on the HPL benchmarks. Built by Fujitsu, the ABCI uses NVIDIA Tesla V100 GPUs and Intel Xeon Gold processors. (Image courtesy of AIST.)

Pictured above is the SuperMUC-NG, ninth fastest supercomputer in the world. It scored 19.5 petaflops and was installed at the Leibiniz Supercomputing Center in Germany. It was built by Lenovo and runs on Intel Platinum Xeon processors. (Image courtesy of the Leibiniz Supercomputing Center.)

This is the 10th fastest supercomputer in the world. Known as the Lassen supercomputer, it was installed at Lawrence Livermore National Laboratory. Its counterpart is the Sierra system. Both supercomputers use the IBM POWER9-NVIDIA V100 GPU architecture. (Image courtesy of Lawrence Livermore National Lab.)

These supercomputers are impressive without a doubt, but they require huge amounts of hardware and are intensely power consumptive. Plus, they are not designed from the ground up with AI applications in mind.

A Common Approach Yields the “Fastest AI Computer”

Cerebras Systems unveiled the CS-1, a computer powered by the biggest chip in the world. According to Cerebras, the CS-1 consumes 17 kilowatts of power, takes up a third of a standard rack, but has machine-learning capabilities equal to hundreds of GPU-stacked racks that run supercomputers. One is already installed and in use at Argonne National Labs (where the first exascale computer in the U.S. is going to be built). Another CS-1 will be installed in the near future at Lawrence Livermore National Laboratory.

The CS-1 speeds up the training of new and massive neural networks, which is a very time-consuming task. But it has an unbelievable piece of hardware: a 1-trillion-transistor wafer-scale processor chip with 400,000 cores.

Cerebras Systems CS-1 blown out illustrates the engineering emphasis on cooling and powering the massive wafer-scale chip that has 1 trillion transistors and 400,000 cores. (Image courtesy of Cerebras Systems.)

The software optimizes the distribution of computational tasks across multiple computers, and up to 32 CS-1s can be linked together to work as a supercomputer. And contrary to GPU-based clusters, each linked computer multiplies the effectiveness of performance at a 1:1 ratio. When 32 CS-1s are linked together, performance increases 32 times. This is not true of GPU clusters, which act like separate small machines instead of one huge computer.

The first task for the CS-1 comes from the U.S. Department of Energy as part of a collaboration with the National Cancer Institute. Their mission is to use the CS-1 to understand and predict how cancer drugs will respond in patients. It will also be used by scientists and researchers to understand the effects of colliding black holes and the resulting gravitational waves they generate.

Bottom Line
There are many AI applications that run on linked machines that are available for use through cloud computing. If supercomputers can be retrofitted to accommodate AI applications and perhaps even designed with them in mind, they will likely help push the boundaries of innovation and discovery in fields like medicine, climatology, physics and space exploration.