This New Logic Gate Network Reduces Inference Speed to Only 4 Nanoseconds

Stanford researchers Felix Petersen and Stefano Ermon, among others, have introduced convolutional differentiable logic gate networks (LGNs) with logic gate tree kernels in their latest research paper published in November 2024. 

This new research integrates a number of concepts from machine vision into differentiable logic gate networks. The research claims to allow for the training of deeper LGNs than before by introducing residual initialisations which help preserve information in deeper networks and prevent vanishing gradients. 

They also introduced logical or pooling which when combined with logic tree kernels, improved training efficiency substantially. 

This could be useful for applications that require both high performance and explainable decision-making, such as robotics, program synthesis, and cognitive modelling.

Improved Architecture

The researchers propose a CIFAR-10 architecture called ‘LogicTreeNet’ which significantly decreases the model size as compared to the SOTA while improving accuracy. 

Peterson also took to X to announce this research recently. “We reduce model sizes by factors of 29x-61x over the SOTA,” he said.

Excited to share our NeurIPS 2024 Oral, Convolutional Differentiable Logic Gate Networks, leading to a range of inference efficiency records, including inference in only 4 nanoseconds 🏎️. We reduce model sizes by factors of 29x-61x over the SOTA.
Paper: https://t.co/Aptk35mKir pic.twitter.com/7nwcZ8PbTB

— Felix Petersen (@FHKPetersen) November 11, 2024

Further, the inference stack demonstrates that convolutional LGNs can be efficiently executed on hardware. 

The paper claims that their model improves accuracy on MNIST while achieving 160x faster inference speeds, but on CIFAR-10, the model improves inference speed by 1900x over the state-of-the-art.

This model combines deep learning strengths with logic gates’ interpretability, enabling networks that can perform logical reasoning. It allows for end-to-end training, enhancing both power and clarity.

By building networks directly from logic gates like NAND, OR, and XOR, it optimises for hardware alignment. This results in faster, more energy-efficient computation, ideal for many applications.

This approach shines in resource-limited environments, like mobile devices and embedded systems, where energy and speed matter. It makes deep learning feasible in settings with limited processing power.

Going Forward

A very intriguing forte for future research in this domain involves applying convolutional differentiable LGNs to computer vision tasks. The researchers express that this would focus on tasks that require continuous decision-making, such as object localisation. 

While LGNs have demonstrated efficiency in tasks like image classification, their potential in handling continuous outputs remains largely unexplored. Investigating this could lead to more efficient and interpretable models for complex vision applications.