Machine learning enthusiasts always have a craze for Computer Vision. Practical Computer Vision tasks require basic knowledge and skills of image processing. Image processing is carried out in different stages of Computer Vision applications such as preprocessing images, deep learning modeling and post-processing. Image processing is extensively used in video datasets compared to image datasets. Image processing finds a crucial place in the deep learning domain with the growing size of image and video data and the increase in digital solution needs.
OpenCV is one of the famously used open-source Python libraries meant exclusively for Computer Vision. Modules and methods available in OpenCV allow users to perform image processing with a few lines of codes. In this tutorial, we discuss different image processing techniques of OpenCV with hands-on Python codes.
This article assumes that the users are familiar with the basics of the OpenCV library. The following articles give a quick kick-start to fulfil the prerequisites of this tutorial.
Sign up for your weekly dose of what's up in emerging technology.
Thresholding on Image
Thresholding is the process of forcing a certain or all the pixel values either to zero or to the maximum possible value. Various thresholding techniques are available to process images. Thresholding can be performed either with a grayscale image or a colour image.
Binary thresholding makes pixel values either to zero or 255 based on the threshold value provided. The pixel values below the threshold value are set to zero, and the pixel values above the threshold value are set to 255. We can recall that 0 refers to black and 255 refers to white in a grayscale image. Thus a grayscale image applied with binary thresholding will become a black-and-white-only image. On the other hand, a colour image applied with binary threshold values will only have black, blue, green, red or any combination of the latter three colours. For instance, combination B+G+R gives white colour, and R+G gives yellow colour.
Inverse binary thresholding is the inverted process of binary thresholding. The pixel values above the threshold are set to zero and the pixel values below the threshold are set to 255. The resulting image will become a black-and-white-only image if the input is a grayscale image.
In truncated thresholding, the pixel values below the threshold are left unaltered, and all other values are set to the threshold value.
Threshold-to-zero is the thresholding process of setting the pixel values below the threshold to zero while the pixel values above the threshold are left unaltered.
Threshold-to-zero-inverse is the thresholding process of setting the pixel values above the threshold to zero while the pixel values below the threshold are left unaltered.
For better understanding, we perform all the above-discussed thresholding processes on a grayscale image.
# read an image in grayscale img = cv2.imread('daria.jpg', 0) img = cv2.resize(img, (320,225)) # apply various thresholds val, th1 = cv2.threshold(img, 110, 255, cv2.THRESH_BINARY) val, th2 = cv2.threshold(img, 110, 255, cv2.THRESH_BINARY_INV) val, th3 = cv2.threshold(img, 110, 255, cv2.THRESH_TRUNC) val, th4 = cv2.threshold(img, 110, 255, cv2.THRESH_TOZERO) val, th5 = cv2.threshold(img, 110, 255, cv2.THRESH_TOZERO_INV) # display the images cv2.imshow('Original', img) cv2.imshow('THRESH_BINARY', th1) cv2.imshow('THRESH_BINARY_INV', th2) cv2.imshow('THRESH_TRUNC', th3) cv2.imshow('THRESH_TOZERO', th4) cv2.imshow('THRESH_TOZERO_INV', th5) cv2.waitKey(0) cv2.destroyAllWindows()
The fixed threshold does not yield good results when there is a difference in illumination on the object of interest. For instance, in the above binary thresholding process, the portion of the human face with more illumination is represented by white pixels, while the portion with less illumination is represented by black pixels. This issue can be tackled by incorporating adaptive thresholding. Adaptive thresholding determines the threshold values locally. It takes as arguments the number of local neighborhood pixels (an odd-valued kernel) and the constant pixel value by which each pixel is subtracted from. The two types of adaptive threshold determination are Mean (arithmetic mean of neighborhood pixels) and Gaussian (Gaussian mean to exclude noises).
img = cv2.imread('daria.jpg', 0) img = cv2.resize(img, (320,225)) # apply various adaptive thresholds th1 = cv2.adaptiveThreshold(img, 255, \ cv2.ADAPTIVE_THRESH_MEAN_C, \ cv2.THRESH_BINARY, 7, 4) th2 = cv2.adaptiveThreshold(img, 255, \ cv2.ADAPTIVE_THRESH_GAUSSIAN_C, \ cv2.THRESH_BINARY, 7, 4) # display the images cv2.imshow('ADAPTIVE_THRESHOLD_MEAN', th1) cv2.imshow('ADAPTIVE_THRESHOLD_GAUSSIAN', th2) cv2.waitKey(0) cv2.destroyAllWindows()
Image smoothing is an important image processing technique that performs blurring and noise filtering in an image. It finds applications in preprocessing and postprocessing of deep learning models. In general, smoothing is performed by a 2D kernel of a specific size on each channel of the image. The kernel average of neighborhoods yields the resulting image. The famous smoothing algorithms in use are Blur, Gaussian Blur, Median Blur and Bilateral Filter.
Blur method is the simple filter that homogeneously applies its kernel to calculate the local weighted average.
img = cv2.imread('sharon.jpg', 1) img = cv2.resize(img, (300,300)) # Apply blur img1 = cv2.blur(img,(3,3)) # display the images cv2.imshow('Original', img) cv2.imshow('Blur', img1) cv2.waitKey(0) cv2.destroyAllWindows()
Gaussian Blurring is the process of removing Gaussian noise from an image. This method takes the image, the kernel size and the standard deviation as arguments.
img = cv2.imread('keiron.jpg', 1) img = cv2.resize(img, (320,210)) # Apply Gaussian blur img1 = cv2.GaussianBlur(img,(5,5),2) # display the images cv2.imshow('Original', img) cv2.imshow('Gaussian', img1) cv2.waitKey(0) cv2.destroyAllWindows()
Median blur is known for its salt-and-pepper noise removal. Unwanted small white and black dots on an image is removed with this tool.
img = cv2.imread('tree.png', 0) # Apply median blur img1 = cv2.medianBlur(img,3) # display the images cv2.imshow('Original', img) cv2.imshow('Median', img1) cv2.waitKey(0) cv2.destroyAllWindows()
Bilateral Filter is used when there is a need for both noise filtering and edges retention. This method detects sharp edges and keeps it as such without any blur.
img = cv2.imread('keiron.jpg', 1) img = cv2.resize(img, (320,210)) # Apply Bilateral Filter img1 = cv2.bilateralFilter(img,7,100,100) # display the images cv2.imshow('Original', img) cv2.imshow('Bilateral', img1) cv2.waitKey(0) cv2.destroyAllWindows()
Image gradients are the rate of change of pixel values in either x or y direction or in both x and y directions. It helps identify the sudden changes in pixel values. In other words, it helps detect the edges. If the gradient is applied in x-direction, vertical edges are detected. If the gradient is applied in y-direction, horizontal edges are detected.
In OpenCV, Laplacian gradient detects edges both horizontally and vertically, while Sobel gradient detects edges either horizontally or vertically.
import numpy as np img = cv2.imread('chessboard.jpg', 0) img = cv2.resize(img, (300,200)) # Laplacian image gradient lap = np.uint8(np.absolute(cv2.Laplacian(img,cv2.CV_64F, ksize=1))) # display the images cv2.imshow('Original', img) cv2.imshow('Lpalacian', lap) cv2.waitKey(0) cv2.destroyAllWindows()
img = cv2.imread('chessboard.jpg', 0) img = cv2.resize(img, (300,200)) # Sobel image gradient vertical = np.uint8(np.absolute(cv2.Sobel(img,cv2.CV_64F, 1,0, ksize=1))) horizon = np.uint8(np.absolute(cv2.Sobel(img,cv2.CV_64F, 0,1, ksize=1))) # display the images cv2.imshow('Vertical', vertical) cv2.imshow('Horizontal', horizon) cv2.waitKey(0) cv2.destroyAllWindows()
Vertical and horizontal edges can be blended together to generate an image gradient with all edges.
Sobel = cv2.bitwise_or(vertical, horizon) cv2.imshow('Sobel', Sobel) cv2.waitKey(0) cv2.destroyAllWindows()
Canny Edge Detection
Canny Edge Detection is a powerful edge detection algorithm that performs Gaussian noise filtering, Sobel based horizontal and vertical edges detection, non-maximum suppression to remove unwanted edge points and hysteresis thresholding with two limiting thresholds to have thin and strong edges.
img = cv2.imread('chessboard.jpg', 0) img = cv2.resize(img, (450,300)) def null(x): pass # create trackbars to control threshold values cv2.namedWindow('Canny') cv2.resizeWindow('Canny', (450,300)) cv2.createTrackbar('MIN', 'Canny', 80,255, null) cv2.createTrackbar('MAX', 'Canny', 120,255, null) while True: # get Trackbar position a = cv2.getTrackbarPos('MIN', 'Canny') b = cv2.getTrackbarPos('MAX', 'Canny') # Canny Edge detection # arguments: image, min_val, max_val canny = cv2.Canny(img,a,b) # display the images cv2.imshow('Canny', canny) k = cv2.waitKey(1) & 0xFF if k == ord('q'): break cv2.destroyAllWindows()
By changing the minimum and maximum threshold values, we can improve the results.
Image contours are the continuous shape outlines present in an image. OpenCV detects the contours present in an image and collects its coordinates as a list. The collected contours can be drawn over the original image back.
img = cv2.imread('valerie.jpg', 1) img = cv2.resize(img, (320,480)) # show original image cv2.imshow('Original', img) # binary thresholding gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) val,th = cv2.threshold(gray, 127,255,cv2.THRESH_BINARY) # find contours contours,_ = cv2.findContours(th, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE) # draw contours on original image # arguments: image, contours list, index of contour, colour, thickness cv2.drawContours(img, contours, -1, (0,0,255),1) cv2.imshow('Contour', img) cv2.waitKey(0) cv2.destroyAllWindows()
Contours are the fundamental building blocks for object shape detection, motion detection and image segmentation. Contours are collected in a Python list. We can draw a particular contour or contours by indexing or slicing those contours alone.
# slice the contours list face = contours[455:465] cv2.drawContours(img, face, -1, (0,0,255),1) cv2.imshow('Contour', img) cv2.waitKey(0) cv2.destroyAllWindows()
In this tutorial, we discussed how image processing is performed with OpenCV in Python. We studied the following methods with hands-on codes and visualizations:
- Simple Thresholding on Images
- Adaptive Thresholding on Images
- Image Smoothing
- Image Gradients
- Canny Edge Detection
- Image Contours
With a basic understanding of image processing, users can dive deeper into real-world Computer Vision problems.
Note: The open-source images used in this article are licensed to reuse without any permission.