Deep Learning Algorithms Are Performing Calculations At The Speed Of Light

Image Credit: Paulius Malinovskis/Flickr

Hardware forms the core aspect of AI applications. Apart from computing power, an efficient and quick mode of relaying information between network layers is equally important. This is where optical techniques are now explored by researchers. In fact, these optical systems can be aggregated into hardware elements such as GPUs.

This article discusses a particular research study by scholars from The University of California, Los Angeles (UCLA), where the team designed neural networks for two tasks, handwritten digit recognition and as an imaging lens.

An Optical Approach

The first record of a scientific study which had optical components for neural networks was in 2004. This study focussed on using a particular tool called matrix gratings for recognition through the Hopfield network. Later, other studies looked at the physical implementations which were conceived as programmable array logic devices or even as analogue computers.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

However, these did not progress well and stalled for a while in research before one novel study tried exploring deep learning under optical systems. This research paper titled All-Optical Machine Learning Using Diffractive Deep Neural Networks, Xing Lin and team from UCLA construct and implement a physical neural network itself based on optical phenomena such as diffraction and interference. Termed as ‘diffractive deep neural network (D2NN)’, each neuron in the network layer is optically connected to another neuron in the subsequent layer with light waves.

d2nn-2
3D Printed D2NN as Imaging Lens (Image courtesy: Xing Lin and team, UCLA)

These are modulated with respect to amplitude and phase created by interference patterns of the previous layers as well as transmission/reflection coefficient, which form the ‘bias’ component of this network and influence the overall learning rates. D2NN is physically constructed through techniques of 3D printing and lithography. When tested, this framework functions at the speed of light in handling learning tasks efficiently.

Network Architecture And Implementations

As mentioned earlier, since D2NN works on diffraction, the network architecture consists of transmissive layers placed between the input and output layers. Every point in the transmissive layer is a neuron having a transmission coefficient. In fact, it is these coefficients that are trained to perform learning tasks at the speed of light.

An important factor taken here is when the secondary light waves pass through this architecture and undergo diffraction, the amplitude in the transmission coefficient of a neuron is assumed to be constant i.e, only phase term is the focus of study. So, this is achieved through error back-propagation algorithm based on stochastic gradient descent (SGD). A detailed account can be found here.Once the setup is ready and a physical model is obtained through a 3D printer, D2NN is trained and tested as a classifier as well as an imaging lens separately.

D2NN architecture with diffractive transmissive layers inserted between the input and output layer (Image courtesy: Xing Lin and team, UCLA)

Handwritten Digit Classification

In this implementation, the neural network acts as a classifier for classifying handwritten digits. So, the 5-layered, 3D-printed D2NN is trained on approximately 55,000 images from MNIST handwritten digit database. Following the training process, this network is tested for 10,000 random images from the same MNIST dataset but those that were not considered in training.

An accuracy of 91.75 percent was achieved in this model. In addition, the network architecture focused the energy from the secondary waves efficiently into the detector region.

Imaging Lens

Another 3D-printed D2NN is now made to perform the task of producing unit-magnified image of the input. The training process used approximately 1500 images from ImageNet database. The underlying principle here is the trained network connects only amplitude in the neurons and layers which is again transmitted at the output.

Once trained this way, the network is tested for different objects illuminated by continuous wave radiation. This D2NN performed very well and correctly projected magnified images without much aberrations in the image.

Conclusion

D2NN is certainly a good start in achieving an exponential speed in executing machine learning at scale. But remember, researchers in this study had to use a powerful optical setup (3D printing and lithography) to run the network altogether. So, this is expensive and acts as a limitation for intelligent systems if deployed on a commercial scale. Only further research can tell the state of optical methods in deep learning.

More Great AIM Stories

Abhishek Sharma
I research and cover latest happenings in data science. My fervent interests are in latest technology and humor/comedy (an odd combination!). When I'm not busy reading on these subjects, you'll find me watching movies or playing badminton.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

Conference, in-person (Bangalore)
Cypher 2023
20-22nd Sep, 2023

3 Ways to Join our Community

Whatsapp group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our newsletter

Get the latest updates from AIM