The primary issue that most underwater photographers face is the loss of color and contrast caused due to the water selectively absorbing and scattering light at different wavelengths. The images are typically clouded by a greenish or blueish color cast, and faraway images are occluded by backscatter. But, engineer and oceanographer, Derya Akkaynakan has developed an algorithm called Sea-thru that, according to her, can “remove the water” from an underwater photograph. It is a physics-based color reconstruction algorithm designed for underwater RGB-D images.
While ‘removing the water’ is a simplified description of the technology for easy understanding, the detailed process and the underlying science behind the algorithm can be read in this paper published by Akkaynak. Presented in June at the IEEE Conference on Computer Vision and Pattern Recognition, this paper was co-authored by engineer Tali Treibitz – Akkaynak’s postdoctoral adviser at the University of Haifa in Israel.
Akkaynak mentions in her research paper, “The Sea-thru method estimates backscatter using the dark pixels and their known range information. Then, it uses an estimate of the spatially varying illuminant to obtain the range-dependent attenuation coefficient.”
Technically, a photograph taken underwater should use a different image formation model than the one derived for the atmosphere, as it fails to factor in the strong wavelength dependency of light underwater. Sea-thru’s image analysis process was developed taking into account the physics involved in the scattering and absorption of light underwater and compares it with the image formation model of the atmosphere. The algorithm then analyses each pixel and calculates the degradation caused by water and reconstructs it effectively by restoring lost colors.
However, another integral parameter for the process is the distance of the subject from the camera. According to Akkaynak, the distance (D) information can come from any source, such as structure-from-motion from multiple images, stereo image pair, specialized sensors, etc.
As part of the process, “more than 1,100 images from two optically different water bodies” were taken, Each of these images were photographed with a color chart kept near the subject in order to “set the scale in the scene.”
Akkaynak explains that she places the color chart at the base of the underwater subject and swims away about 15 meters from the subject. She then swims towards the subject, photographing it from different distances and angles, even from the top. This set of data provided the necessary distance information required to develop the image formulation model for Sea-thru.
She also clarified that this algorithmic process is not the same as photoshopping an underwater image wherein a user can enhance the image by dehazing and pumping up the colors. “It’s a physically accurate correction, rather than a visually pleasing modification,” states Akkaynak.
The Sea-thru algorithm can prove to be immensely beneficial for scientists studying marine ecology and the impact of climate change, pollution and overfishing on the coral reefs and other marine systems.
The Missing Connecting Link Between AI & Marine Research
When it comes to marine research, AI and machine learning have not played a crucial role in fueling important discoveries, when compared to other research domains, even though there’s an entire goldmine of data waiting to be unraveled.
In an exclusive interview with Analytics India Magazine, Akkaynak mentions, “Scientists and research labs routinely use consumer cameras to study important questions about the ocean. For example, a diver goes out to survey a coral reef, and comes back with hundreds of photos; an autonomous or remotely operated robot goes out for a mission and comes back with tens of thousands of images. While we collect vast amounts of imagery throughout the world, we only manage to analyze a tiny fraction of those images.”
She goes on to explain that since the underwater images are too degraded for automated analysis, the analysis has to be done manually by a human expert (tedious, slow, and expensive). The interaction of light with the particles in the water causes it to lose many of its colors, leading to dull, dark, monochromatic images. Because of this, AI designed to work on images taken in air performs poorly on underwater images. Also, an AI trained on one underwater image dataset will not work consistently or effectively on another dataset. “Thus, on average, it takes a human expert 2 hours to analyze an underwater video that is 1 hour long,” she adds.
According to Akkaynak, Sea-thru is the advent of the boom of artificial intelligence in marine science. “With Sea-thru, we will be able to reverse the degradation in the images due to the presence of the water, and also standardize underwater scenes to what would have been their appearance on air. Then, we will be able to use the full power of computer vision and machine learning algorithms on large underwater datasets, thereby speeding up the pace at which we study marine sciences, and learn about our oceans,” she told Analytics India Magazine.
