You may have used or seen filters in Instagram reels of cartooned people or colouring backgrounds or smoothing the face while taking the picture on your camera. The general meaning of filter is modifying the existing image into another using kernels. There are a variety of filtering techniques used for processing the images, each has its own specific cases. Guided image filtering is one of such techniques which performs edge-preserving smoothing on an image, using the content of another image. In this article, we will discuss guided image filtering in detail along with working and real-life use cases. The major points to be discussed in this article are listed below. 

Table of Contents

1. What are Guided Image Filters?
2. Bilateral filters
3. Difference between Bilateral and Guided Filters
4. How does Guided Image Filtering work?
5. Applications of Guided Image Filtering
6. Advantage and Disadvantage of Guided Image Filtering
7. Real-life Use-Cases of Guided Image Filtering

Let’s start the discussion by understanding what guided image filters are.

What are Guided Image Filters?

Guided image filters are mainly used for edge detection which uses an edge-preserving smoothing filter for detecting the edges. It has a local linear model which means a model without bias. It assumes that at a particular point, the only places that will be affected are from all the pixels inside a box. Input image uses the content of the second image called the guidance image. A guidance image can be the input image itself of a version of the same image.

In the image, we can see an input image, a guidance image and an output filtered image. The operation performed per pixel is output at any pixel is basically a result of multiplying each pixel in the input image with some sort of weight that we will generate using a guidance image for that particular pixel. This operation is done to every pixel of the image as:

The above equation is the function of guidance image I, where q is the output, p is input, W is weight, and i, j are the pixel indexes. 

Bilateral filters

A bilateral image filter is a non-linear, noise-reduction smoothing and edge-preserving filter for images. This filter weighted averages the nearby pixels resulting in the change in intensity of each pixel. In the below image, we can see smoothing and noise reduction on the image.

Image source

(Left image is the original image and right image is the result of bilateral filtering)

Difference between Bilateral and Guided Filters

Bilateral filters use nonlinear, noise-reducing smoothing and edge-preserving filters. The output of the bilateral filter uses a weighted average of the nearby pixels. Whereas the guided filter uses a local linear model as an edge-preserving filter. The bilateral filter suffers from gradient reversal artefacts that mean the introduction of false edges in the image.

Image source

Image source

Another major difference between bilateral and guided filters is performance. As we can see in the above graph, the increase in radius (means the influence of filter or area covered by filter) causes the time for the filter to smooth the image also increase in the case of Bilateral filter. But in the case of a Guided filter, it is instant or independent of the size of the image.

How Does it Work?

Consider that our input image as I, output image as q, a guidance image between the input image, and w_k as window centred at the pixel k. The output is expressed as a linear transformation:

Where a_k and b_k are constants and linear coefficients. Form input p_i and output q_i to reduce the error and to determine coefficients a_k and b_k we use cost function as:

Where Ɛ is the regularization parameter. Here,

Where |w| is the number of pixels, 𝞵 _k and σ ²_k is mean and variance of I, respectively.

Introducing the linear model to all local windows over the entire image and averaging the acquired values of q₁ yields the filter.

Applications of Guided Image Filters

Some important applications of guided image filtering are listed below:-

• Edge-preserving smoothing: Here edge-preserving smoothing filter can preserve the key features of an image like an edge and denoise the image.
• Flash/No-Flash Denoising: It can denoise a no-flash image under the guidance of its flash version.
• Matting/Guided Feathering: Extracting foreground objects from an image that is means separating foreground from background. It is used in video editing and image processing.
• Haze Removal: Hazey images are formed due to light scattering with particles in the atmosphere. Haze removal filters will improve the image.
• Joint Upsampling: Under the guidance of another image, upsampling are done. One application of joint upsmapling is the colorzation of images.

Advantages and Disadvantage of Guided Image Filtering

The major advantages and disadvantages of guided image filtering are listed below:-

Advantages

• Guided image filters perform well in terms of both quality and efficiency in a wide variety of applications such as noise reduction, detail smoothing etc.
• No Gradient reversal in guided image filter resulting in the distinguishable edge.
• Guided image filter’s performance in terms of speed of implementation is way better than the bilateral filters.

Disadvantage

The only disadvantage of guided image filtering is that without a clear and/or sharp guidance image, the guided filter may fail. 

Real-Life Use Case of Guided Image Filtering

Guided filters have many applications, as it is mentioned above. A guided filter is used in digital image restoration by removing noise from the image. Guided filters are used with colour images also. In the case when the filtering input is multichannel, it is straightforward to apply the filter to each channel independently. Let us understand this with the below applications.

 (a) Input HDR image, (b) Compressed image using the O(N) bilateral filter, (c) Compressed image using the guided filter

Final Words

Through this article, we have seen why filters are used in image processing. We could also understand how guided image filters work with their real-life applications. We have also gone through the difference between bilateral and guided image filters and listed how they work differently. 

References

• Guided Image Filtering
• Self-guided filter for image denoising