Superman can see through walls, but I am not sure whether he can see things that are not in his line of vision – around the corners. But thanks to a new technology developed by a team of researchers from Princeton, Stanford, Southern Methodist, and Rice Universities, we might be able to do so.
The team has proposed a new non-line-of-sight (NLoS) imaging system that uses lasers and deep learning to see objects around corners in real-time. The new imaging system uses a commercially available camera sensor and a laser source that is as powerful as the one found in a laser pointer. Bouncing off from the visible wall, the continuous-wave (CW) laser beam illuminates the hidden object, which then reflects it onto the wall. This creates an interference pattern on the wall known as a speckle pattern that encodes the shape of the hidden object.
To reconstruct the hidden image from the speckle pattern, the team looked to AI. However, real-time imaging required short exposure times, which gave rise to too much noise, thereby restricting the applicability of the existing algorithms. The team, therefore, had to adopt deep learning for the optical reconstruction process.
Subscribe to our Newsletter
Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.
“AI/deep learning allows us to reconstruct the hidden object using low light (i.e. noisy) measurements,” Dr Christopher A. Metzler from Stanford University and Rice University, and the research team leader told Analytics India Magazine.
“By accurately characterising the noise, we were able to synthesise data to train the algorithm to solve the reconstruction problem using deep learning without having to capture costly experimental training data,” co-author Prasanna Rangarajan from Southern Methodist University explained.
The research has been published in Optica – The Optical Society’s journal for high-impact research.
The Differentiating Factors
This is not the first non-line-of-sight (NLoS) imaging system that researchers have proposed. In one of the studies, researchers from Intel Labs and Stanford University proposed the acoustic non-line-of-sight imaging that uses readily available, low-cost microphones and speakers to image and resolve 3D shapes hidden around corners.
Talking about how their system is different from the other approaches, Dr Metzler told Analytics India Magazine, “Ours improves upon the resolution and frame rates of existing systems. Traditional NLOS systems use time of flight information, which limits their resolution to about a cm. Ours uses interference patterns, which allow for much higher resolution.”
“Likewise, because it is not using time-of-flight information, our system can use sensors that collect more light. We also developed deep learning algorithms that allow us to operate with even less light,” he added.
According to him, these attributes allow the system to perform operations such as reading the license plate of a hidden car as it is driving or reading a badge worn by someone walking on the other side of a corner.
The researchers tested this approach by trying to reconstruct the images of even minute hidden objects such as 1-centimetre-tall letters and numbers hidden behind a corner. The imaging setup was positioned about 1 meter from the wall. This NLOS approach produced reconstructions with a resolution of 300 microns, using just two 1/8th exposure-length images from a standard complementary metal-oxide-semiconductor detector.
Dr Metzler explains that the reason this approach operates using standard and relatively low-cost sensors are that it does not rely on time-of-flight information. This is another factor that differentiates this approach from other NLOS imaging systems that generally require highly-sensitive cameras and sensors.
Since the team could not use existing AI algorithms to reconstruct the hidden image from the speckle pattern, they had to use a specially-trained deep-learning algorithm. “Compared to other approaches for non-line-of-sight imaging, our deep learning algorithm is far more robust to noise and thus can operate with much shorter exposure times. By accurately characterising the noise, we were able to synthesise data to train the algorithm to solve the reconstruction problem using deep learning without having to capture costly experimental training data,” said Rangarajan.
Real-World Applications
This research is part of the U.S. Defense Advanced Research Projects Agency’s (DARPA’s) Revolutionary Enhancement of Visibility by Exploiting Active Light-fields (REVEAL) program. This program deals with developing and fostering new technologies to image hidden objects around corners.
“Non-line-of-sight imaging has important applications in medical imaging, navigation, robotics, and defence. Our work takes a step toward enabling its use in a variety of such applications,” said co-author Dr Felix Heide from Princeton University.
“Our system is still lab-bound. However, with further development, it could be used for detecting pedestrians and other cars around corners when driving, search and rescue in disasters, surveillance, and remote sensing,” Dr Metzler told Analytics India Magazine.
The researchers are still working on refining the process to enhance the practical quotient of the system and widen its scope of real-world application by extending the field of view so that it can reconstruct larger objects.
