IIT-Bombay’s energy-saving AI chip accomplishes new ultralow-power lows

A research team from IIT-Bombay has developed a highly efficient AI chip.. The team consists of Maryam Shojaei Baghini (Professor IIT-B, Senior Member IEEE), Udayan Ganguly(Professor IIT-B, Senior Member IEEE), Ajay Kumar Singh(pursuing PHD at IIT-B) and Vivek Saraswat(pursuing PHD at IIT-B).

Spiking Neural Networks

The biological brain works on real-time signals (audio or video), which are slow (micro seconds), but the switching time of conventional CMOS is fast (nanoseconds). The fast CMOS-based chip 'guzzles' power at high speed, waiting for the slow real-time signal. Spiking neural networks (SNNs) are the third generation of artificial neural networks. ​​They emulate the original language of intelligence as opposed to AI chips of today that create artificial strategies (inspired by mostly mathematics, statistics) to make sense of the deluge of natural data. However, SNNs process signals differently from microprocessors that use digital representation. Hence SNNs work very inefficiently on standard chips. This is why there is a push to develop custom brain-like hardware.

“All natural signals arrive in time. Hence, biology uses time as information. Naturally, neurons produce spikes where the "time of spike" is used to encode and process all information. It is a complete language that represents all our thoughts, sensations and actions. It also represents learning, the emergence of creativity and problem solving, emotions,” said Udayan Ganguly.

Image: SNN architecture versus a biological neuron

SNN works with discrete events that happen at defined times instead of working with continually changing time values. The secret to efficiency in both size and power lies in the construction and language of information processing in the brain.

Quantum tunnelling

Quantum tunnelling is a phenomenon whereby a wavefunction can propagate through a potential barrier. This quantum phenomenon is key to the creation of the chip, as quantum tunnelling removes the need to create enough charge to overcome the potential barrier.

A simple neuron model is called the Leaky Integrated and Fire (LIF) model by neuroscientists. It is modelled as a current being used to charge up a capacitor, thereby producing an integration function. The capacitor can be leaky. When the charge stored (i.e. voltage) on the capacitor reaches a threshold, the spike is issued, and the capacitor charge is reset to zero.

Typically transistors are operated in on-state (say ~ milliamperes of current) which guzzles power. The team proposed the tunnelling current between the source/drain and the floating body terminal of the MOSFET as the ultra-low current source. This is akin to operating the transistor at the off-current levels enabling very low power/energy.

The tunnelling current (i.e. current in the off state) is 10,000x smaller than a transistor's typical on-current (~ milliamperes). Overall, this translates to 5000x less energy per spike compared to a transistor operating conventionally using on-current.

Further, quantum tunnelling also saves up precious silicon area by eliminating the need for large capacitors to slow down the operation to natural information timescale. 

Efficiency

The researchers compared LIF with other analog, mixed signal chips which are more equivalent.

Source: ieeexplore.org

The results revealed the ultra-energy-efficient neuron which consumes 8.2fJ of energy per spike and static power of 2.7pW while occupying only a 40um² area on silicon. Moreover, compared to state-of-art neurons implemented in hardware spiking neural networks, we achieved 5000x lower energy-per-spike at a similar area and 10x lower standby power at a similar area and energy-per-spike. 

"To test a neuron fairly, it must do something useful in hardware. Our neurons were tested in a speech recognition application where the 20 input neurons with spike coded input with 36 neurons "Liquid" reservoir producing neuronal reservoir output from the chip was tested with spike coding, and classification was done off-chip. The results showed promising performance – indicating the feasibility of the neuron for real-world applications for ultra-low power edge applications," said Udayan.

Potential use cases

The AI chip, though still in the prototype stage, showed the ultra-low power neurons can be integrated to form a neurosynaptic core of 36 neurons connected through simple transistor synapses. The neurosynaptic core is like a primitive microprocessor core.

The use cases include voice activity detection and speech classification, gesture recognition, motion pattern recognition, navigation, biomedical signals (e.g. ECG) classification, and signal reconstruction. 

While these applications can be run on a server or a supercomputer, the efficiency in area and power offered by SNNs enables these applications to be ported to edge devices, e.g. mobile phones, IoT sensors, etc.