Artificial intelligence (AI) is a topic of growing importance in the tech world. And while the AI sector is still somewhat frothy—containing fewer players in comparison to other sectors—it has shown sparks of ingenuity that threaten to potentially dictate the future of engineering. One such spark can be found in the recently released Habana Goya inference chip—a 16nm deep-learning chip that the company claims has processing speeds that eclipse those of NVIDIA GPUs.
The Goya inference chip contains eight VLIW cores that are programmable in C and supports 8- to 32-bit floating point and integer formats. According to Habana, the Goya chip can process up to 15,000 ResNet-50 images per second, with a latency of 1.3ms and a batch size of 10, when running at 100W. This, of course, is a far cry from NVIDIA V100’s 2,657 images per second and Intel Xeon Platinum 8180’s 1,225 images per second. When operating at a batch size of one, the Goya chip also reportedly can handle up to 8,500 ResNet-50 images per second with a latency of 0.27ms.
According to the company, these impressive numbers are due to a combination of elements such as the architecture of the chip, a proprietary graph compiler, software-based memory management and mixed-format quantization. However, there are still questions about some of the chip’s features, such as its very small batch size of 10—when compared to NVIDIA’s 256—which calls into question whether the Goya will be able to sustain its impressive performance at higher batch sizes.
Habana has given a demonstration of the Goya chip working with graphs from MXNet and ONNX frameworks. The startup is also working on supporting TensorFlow and, by extension, other major AI frameworks. The company plans to release samples of its 16nm Gaudi chip for neural network training in the second quarter of 2019.
For more exciting product announcements, check out Dell Continues Big Moves in the PC World with New Product Releases.