24th October 2023

New AI chip is 22 times faster

NorthPole, a new edge-based processor announced this month by IBM Research, is up to 22 times faster and much more energy efficient than chips currently on the market.

A team from IBM research has presented NorthPole – a brain-inspired chip architecture, which blends computation with memory to process data more efficiently at low energy costs.

Throughout its history, computing has been processor-centric, with memory separated from compute. However, shuttling huge amounts of data between memory and compute comes at a high price in terms of both energy consumption, as well as processing bandwidth and speed. This is particularly evident with advanced real-time artificial intelligence (AI) applications, such as facial recognition, object detection, and behaviour monitoring, which require fast access to vast amounts of data. As a result, most contemporary computer architectures are rapidly reaching physical and processing bottlenecks and risk becoming economically, technically, and environmentally unsustainable, given the growing energy costs involved.

Inspired by the neural structure of the organic brain, Dharmendra Modha and colleagues developed NorthPole – a neural inference architecture that combines compute with memory on a single chip. Their work is published this month in the journal Science.

According to the study authors, the NorthPole chip "merges the boundaries between brain-inspired computing and silicon-optimised computing, between compute and memory, between hardware and software." Its performance, energy efficiency, and area efficiency are all higher compared to other devices. And because NorthPole works as a digital system, its operation is unaffected by device noise, systemic biases, drifts, and other problems that afflict analogue systems.

NorthPole consists of 256 computing cores, each of which contains its own memory. Fabricated with a 12 nanometre (nm) node process, it packs 22 billion transistors into just 800 square millimetres, the size of a postage stamp. By eliminating the need to frequently access external memory, it performs tasks such as image recognition much faster than existing architectures do, while consuming vastly less power.

Modha and his colleagues tested NorthPole's capabilities on the ResNet50 benchmark image classification network, where it achieved 25 times higher energy metric of frames per second (FPS) per watt, a five times higher space metric of FPS per transistor, and a 22 times lower time metric of latency relative to comparable technologies. It even outperformed those using a 4 nm process, such as NVIDIA's H100 with 80 billion transistors.

"Its energy efficiency is just mind-blowing," said Damien Querlioz, a nanoelectronics researcher at the University of Paris-Saclay in Palaiseau. "I feel the paper will shake the common thinking in computer architecture."

There is one caveat. The new chip is only able to run specialised AI processes, for now. It cannot be used for training processes, or large language models with billions of parameters like ChatGPT. However, IBM's researchers plan to try connecting multiple NorthPole chips together, which they believe will overcome this current limitation. The company says that further gains could be achieved by shrinking its 12 nm process down to 3 nm or possibly smaller, which is the present state of the art for transistors. In the future, NorthPole could be used in autonomous vehicles, robotics, digital assistants, and satellite observations, says IBM.

Comments »

If you enjoyed this article, please consider sharing it: