While it comes with extreme benefits, artificial intelligence has time and again proved to be notoriously energy-consuming when implemented with traditional methods. According to data, training a single AI model can produce the same amount of CO2 as done by five cars in their whole span. In an attempt to create energy-efficient AI, researchers at UCL have created a system that can enhance the performance of a brain-inspired computing system by reducing the AI’s carbon footprint.
In a recent study, published in a paper, the researchers and engineers of UCL have suggested the application of ‘committee machines’ on the memristor-based neural networks. Physically implemented neural networks undergo issues like device faults, random telegraph noise, variability in device-to-device interaction, and line resistance. Thus, it was noted that the accuracy of the AI system could be vastly improved by replacing the transistors with memristors for all devices. Deploying simulations and experimental data from memristive devices would further allow them to enhance their performance without increasing the number of memristors.
With the ever-increasing demand for data, it can cause difficulties in increasing data transmission capacity after a certain point; thus, this study can be extremely beneficial for the current era.
Also Read: Google’s ‘Brahmastra’ For NLP Supremacy
Overview Of The System & Method
Considering the conventional methods come with non-idealities and faulty devices during inference, the researchers suggested combining non-ideal memristor-based neural networks into categories or committees in order to achieve better performance. This method of utilising committee machines (CMs) will not only increase the accuracy of the inference but also enhances the computation time with the parallel working of memristive artificial neural networks (ANNs).
Explaining the process, Dr Adnan Mehonic, a researcher at the department of electronic and electrical engineering of UCL, stated in the paper that to facilitate this they deployed simulation setup, where the memristive ANNs are trained to be simulated and combined into CMs. Continuing the experimentation, the researchers analysed three memristive devices — tantalum/hafnium oxide-based device, tantalum oxide-based device, and amorphous vacancy modulated conductive oxide-based device, based on their non-idealities. And then leveraged the experimental data to simulate and compare the results of memristive ANNs — individual as well as in committees.
Also Read: How Open-Sourcing AllenAct Provides A Substantial Growth For Embodied AI
How Does It Work?
Starting with the simulation set up, the researchers used the MNIST database to train the fully connected ANNs to identify handwritten digits. Following that, the weights of ANNs were categorised in pairs using proportional mapping scheme, which has simulated the memristive-based ANNs. Finally, these neural networks were distributed using experimental data to check on the non-idealities and then combined into CMs.
Utilising multiple neural networks to enhance the accuracy of inference
With this, researchers highlighted that the method of CM, such as ensemble averaging, can be used with traditional ANNs to produce better performance and accuracy. Further, according to researchers with ensemble averaging, the memristive ANNs use the same digital weights as they were mapped before. Thus the neural networks in committees are performing better than the ones working individually.
Further, experiments with tantalum/hafnium oxide-based memristive devices provide a comprehensive picture of the non-idealities. It used line resistance effects to analyse the faulty devices. This sort of device doesn’t exhibit apparent telegraph noise, and therefore usually preferred for inference application.
Experimental data of crossbar for Ta/HfO2 RRAM memristive device
Additionally, to explore the result of reducing the adverse effect of random telegraph noise, another memristor technology has been used — tantalum oxide-based. Although to investigate the telegraph noise measurements from a single memristive device has been considered, for simulating line resistance results, interconnect resistance from tantalum oxide-based devices is utilised.
Simulating telegraph noise for the tantalum oxide-based device
The researchers have also considered the ‘amorphous vacancy modulated conductive oxide-based memristive device’ to check the effect of random telegraph noise. With all these methods, the researchers noted that —tantalum/hafnium oxide-based memristive device produced a median accuracy of ~95.7%; tantalum oxide-based device had median accuracy of ~96.5%; the third one gave median accuracy of ~96.7%.
Also Read: How Microsoft & Amazon Plan To Go Carbon Negative
Conclusion
These results mentioned above showcased how the method of involving committee machines have drastically improved the accuracy of memristive ANNs, despite its non-idealities. It has also been observed that the accuracy of individual neural networks hugely determines the accuracy of the ones in CMs. Thus, there is always a chance to enhance the performance of the devices and systems that implement these networks.
This method and system will not only enhance the performance and accuracy of the system but will also boost up the robustness, flexibility and reduce the carbon footprint of the machine. It is now the perfect time for memristors to take the lead in developing more energy-sustainable ANNs on IoT devices as well as edge computing, concluded researchers at UCL.
Read the whole paper here.
