In today’s era, images play a vital role in cognitive development as visuals hold more appeal than plain text to curious and intuitive minds. Visuals help improve learning enormously and on multiple levels. Studies have shown that around 65% of the population constitute visual learners. Due to this reason, study and research revolving around images, such as image processing, image recognition, and image blending, have created a buzz among masses of people developing new technologies. Image processing programs such as Photoshop and other freeware alternatives such as GIMP and Paint.net offer a wide range of possibilities to edit and manipulate images and photos. Our idea would be to explore the domain further, to discover what more can be done to perform operations that are not typically built in existing software and find new ways to teach and educate oneself.
What is Image Blending?
Image Blending is mixing two images of the corresponding pixel values to create a new target image. The concept of blending images is comparatively very easy. To achieve this, we can simply make a copy of an image and transfer each source pixel’s values into a pixel in the target image. With the help of Python and the OpenCV library, it becomes relatively very easy to work with visual inputs such as images or videos. The OpenCV library contains most of the functions that we would require for working with images. Using OpenCV, we can combine the photos, we combine layer masks and gradients and explore other numerous possibilities.
The Main Aspects for Image Blending
To start working with Image Blending, we would need to know the bare basics of image processing techniques. Image Blending techniques mainly focus on two major aspects for creating a blended image by using Image Pyramids, namely Gaussian and Laplacian Pyramids. A pyramid or pyramid representation is a type of multi-scale representation where an image is repeatedly smoothened and subsampled. It searches for a target pattern over many repetitive scales. At first from the image, it creates copies of the target pattern and reconstructs it over several expanded scales. Each copy is then convolved with the original image, creating a new and unique image.
The Gaussian Pyramid technique focuses on resizing the image, reducing it to one fourth in every cycle. Its goal is to define the representation of what information in the image is explicitly available at different scales. It has several other applications such as Progressive image transmission, Scale-invariant template matching and Efficient feature search. Here,
Image cross-correlation, convolution and matrix multiplication are the core aspects of this operation. The Laplacian Pyramid technique, on the other hand, keeps track of differences between the multiple sizes of images and stores the information for further use. It does so by gray scaling the image to be processed, focusing on the image outlines at different sizes, and comparing its differences. It also uses a linear transformation to decompose an image into various components by multiplying it with a set of transformation functions. It can be termed as a filter that detects features from the image and stores them in memory for further processing.
Getting Started with Code
Now that we know how the Image Pyramid works, we will try to implement a basic image pyramid for image blending using both Gaussian and laplacian techniques. We have used two images with two different backgrounds on purpose to see how the blending technique comes into play and compare the difference between a merged image and a blended image.
The below code is in reference to the official implementation, whose video tutorial you can find here.
Installing The OpenCV Package and Importing It
To Install OpenCV, you can use the following command :
!pip install opencv-python
Then to import the package and NumPy, use
import cv2 import numpy as np
Now, we will set a path for our images of the directory they are present in.
Here, were are going to blend two images, an image of an apple and an orange.
apple_path = '/content/OtherApple.jpg' orange_path = '/content/Orange.jpg' apple = cv2.imread(apple_path) orange = cv2.imread(orange_path)
Printing The Shape and Size of Our Images Imported
#printing the shape of our images print(apple.shape) print(orange.shape)
We will get the following output
(512, 512, 3) (512, 512, 3)
Creating Left and Right Halves for Merging
Now that we have the sizes of our images let’s divide them into a right and left half to merge them together as one. We’ll use the hstack function to feed them into the system as tuples.
apple_orange = np.hstack((apple[:, :256], orange[:, 256:])) #Dividing them into right and left half as tuples using hstack
Let’s check our results so far. To plot this, we will be using matplotlib.
import matplotlib.pyplot as plt plt.imshow( apple_orange[:, :, ::-1 ])
As we can observe, there is still a visible line in between the two images. However, they have been merged but not blended. Image Blending will now come to play to blend both the images properly.
Steps to create an Image Blender
- First, we will create a gaussian pyramid for both the apple and orange image.
- Next, from the created Gaussian pyramid, further process and find the Laplacian pyramid.
- Rejoin the left half of the apple image and right half of the orange image in each level of Laplacian pyramids
- Blend the merger line and reconstruct the original image
Generating Gaussian pyramid for apple and orange
Here we are using a range as six, which is a very commonly used range. Range defines the number of times you want to reduce the resolution. One can tune this accordingly. Cv2.pyrDown helps us execute the gaussian pyramid.
# generate Gaussian pyramid for apple apple_copy = apple.copy()#create a copy of the apple image gp_apple = [apple_copy] #creating the gaussian pyramid and pass as a list for i in range(6): apple_copy = cv2.pyrDown(apple_copy) gp_apple.append(apple_copy)
Applying the same steps for orange image,
# generate Gaussian pyramid for orange orange_copy = orange.copy() gp_orange = [orange_copy] for i in range(6): orange_copy = cv2.pyrDown(orange_copy) gp_orange.append(orange_copy)
Generating Laplacian pyramid for apple and orange from gaussian
We are going to make use of the 5th element from the Laplacian pyramid. The level in the Laplacian pyramid is created by the difference between Gaussian and extended level to its upper level in the gaussian pyramid. The Laplacian method also makes use of cv2.pyrUp to rescale the image.
# generate Laplacian Pyramid for apple apple_copy = gp_apple[5] lp_apple = [apple_copy] for i in range(5, 0, -1): gaussian_expanded = cv2.pyrUp(gp_apple[i]) laplacian = cv2.subtract(gp_apple[i-1], gaussian_expanded) lp_apple.append(laplacian) #Doing the same from orange, # generate Laplacian Pyramid for orange orange_copy = gp_orange[5] lp_orange = [orange_copy] for i in range(5, 0, -1): gaussian_expanded = cv2.pyrUp(gp_orange[i]) laplacian = cv2.subtract(gp_orange[i-1], gaussian_expanded) lp_orange.append(laplacian)
Merging the Left and Right Halves
Using the zip function, we are merging both the laplacian pyramids for both apple and orange.
# Now add left and right halves of images in each level apple_orange_pyramid = [] n = 0 for apple_lap, orange_lap in zip(lp_apple, lp_orange): n += 1 cols, rows, ch = apple_lap.shape laplacian = np.hstack((apple_lap[:, 0:int(cols/2)], orange_lap[:, int(cols/2):]))#dividing columns of both images into half apple_orange_pyramid.append(laplacian)#appending the variable to list
Reconstruct the Image
The final step is to reconstruct our image. We will first create a variable, create a ‘for’ loop to loop from 1 to 6 range, and add all the pyramid layers created.
# now reconstructing our image using pyrUp and stating pyramid levels apple_orange_reconstruct = apple_orange_pyramid[0] for i in range(1, 6): apple_orange_reconstruct = cv2.pyrUp(apple_orange_reconstruct) apple_orange_reconstruct = cv2.add(apple_orange_pyramid[i], apple_orange_reconstruct)
Loading our final product and Checking Result
#using -1 to maintain coloured saturation plt.imshow( apple_orange_reconstruct[:, :, ::-1 ])
As you may notice now, the merge line has now been blurred, and the image seems blended properly.
EndNotes
By performing the following steps, we have now created and learnt the basics of image processing and blending. We have used such variant images to get a taste and notice the difference in between. You can choose better and properly aligned images to merge, blend, and create your own unique blended image. The full Colab file for the following can be accessed from here.
Happy Learning!
