Advertisement

Complete Guide to Image Processing with OpenCV in Python

Image processing is carried out in all stages of Computer Vision such as preprocessing images, deep learning modeling and post-processing
OpenCV Image Processing

Machine learning enthusiasts always have a craze for Computer Vision. Practical Computer Vision tasks require basic knowledge and skills of image processing. Image processing is carried out in different stages of Computer Vision applications such as preprocessing images, deep learning modeling and post-processing. Image processing is extensively used in video datasets compared to image datasets. Image processing finds a crucial place in the deep learning domain with the growing size of image and video data and the increase in digital solution needs.

OpenCV is one of the famously used open-source Python libraries meant exclusively for Computer Vision. Modules and methods available in OpenCV allow users to perform image processing with a few lines of codes. In this tutorial, we discuss different image processing techniques of OpenCV with hands-on Python codes.

This article assumes that the users are familiar with the basics of the OpenCV library. The following articles give a quick kick-start to fulfil the prerequisites of this tutorial. 

Getting Started with OpenCV in Python

Real-time GUI Interactions with OpenCV in Python

Thresholding on Image

Thresholding is the process of forcing a certain or all the pixel values either to zero or to the maximum possible value. Various thresholding techniques are available to process images. Thresholding can be performed either with a grayscale image or a colour image.

Binary thresholding makes pixel values either to zero or 255 based on the threshold value provided. The pixel values below the threshold value are set to zero, and the pixel values above the threshold value are set to 255. We can recall that 0 refers to black and 255 refers to white in a grayscale image. Thus a grayscale image applied with binary thresholding will become a black-and-white-only image. On the other hand, a colour image applied with binary threshold values will only have black, blue, green, red or any combination of the latter three colours. For instance, combination B+G+R gives white colour, and R+G gives yellow colour.

Inverse binary thresholding is the inverted process of binary thresholding. The pixel values above the threshold are set to zero and the pixel values below the threshold are set to 255. The resulting image will become a black-and-white-only image if the input is a grayscale image.

In truncated thresholding, the pixel values below the threshold are left unaltered, and all other values are set to the threshold value.

Threshold-to-zero is the thresholding process of setting the pixel values below the threshold to zero while the pixel values above the threshold are left unaltered.

Threshold-to-zero-inverse is the thresholding process of setting the pixel values above the threshold to zero while the pixel values below the threshold are left unaltered.

For better understanding, we perform all the above-discussed thresholding processes on a grayscale image.

 # read an image in grayscale
 img = cv2.imread('daria.jpg', 0)
 img = cv2.resize(img, (320,225))
 # apply various thresholds
 val, th1 = cv2.threshold(img, 110, 255, cv2.THRESH_BINARY)
 val, th2 = cv2.threshold(img, 110, 255, cv2.THRESH_BINARY_INV)
 val, th3 = cv2.threshold(img, 110, 255, cv2.THRESH_TRUNC)
 val, th4 = cv2.threshold(img, 110, 255, cv2.THRESH_TOZERO)
 val, th5 = cv2.threshold(img, 110, 255, cv2.THRESH_TOZERO_INV)
 # display the images
 cv2.imshow('Original', img)
 cv2.imshow('THRESH_BINARY', th1)
 cv2.imshow('THRESH_BINARY_INV', th2)
 cv2.imshow('THRESH_TRUNC', th3)
 cv2.imshow('THRESH_TOZERO', th4)
 cv2.imshow('THRESH_TOZERO_INV', th5)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
thresholding on image
thresholding on image

The fixed threshold does not yield good results when there is a difference in illumination on the object of interest. For instance, in the above binary thresholding process, the portion of the human face with more illumination is represented by white pixels, while the portion with less illumination is represented by black pixels. This issue can be tackled by incorporating adaptive thresholding. Adaptive thresholding determines the threshold values locally. It takes as arguments the number of local neighborhood pixels (an odd-valued kernel) and the constant pixel value by which each pixel is subtracted from. The two types of adaptive threshold determination are Mean (arithmetic mean of neighborhood pixels) and Gaussian (Gaussian mean to exclude noises).

 img = cv2.imread('daria.jpg', 0)
 img = cv2.resize(img, (320,225))
 # apply various adaptive thresholds
 th1 = cv2.adaptiveThreshold(img, 255, \
                                  cv2.ADAPTIVE_THRESH_MEAN_C, \
                                  cv2.THRESH_BINARY, 7, 4)
 th2 = cv2.adaptiveThreshold(img, 255, \
                                  cv2.ADAPTIVE_THRESH_GAUSSIAN_C, \
                                  cv2.THRESH_BINARY, 7, 4)
 # display the images
 cv2.imshow('ADAPTIVE_THRESHOLD_MEAN', th1)
 cv2.imshow('ADAPTIVE_THRESHOLD_GAUSSIAN', th2)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
adaptive thresholding on image

Image Smoothing

Image smoothing is an important image processing technique that performs blurring and noise filtering in an image. It finds applications in preprocessing and postprocessing of deep learning models. In general, smoothing is performed by a 2D kernel of a specific size on each channel of the image. The kernel average of neighborhoods yields the resulting image. The famous smoothing algorithms in use are Blur, Gaussian Blur, Median Blur and Bilateral Filter.

Blur method is the simple filter that homogeneously applies its kernel to calculate the local weighted average.

 img = cv2.imread('sharon.jpg', 1)
 img = cv2.resize(img, (300,300))
 # Apply blur
 img1 = cv2.blur(img,(3,3))
 # display the images
 cv2.imshow('Original', img)
 cv2.imshow('Blur', img1)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
blur processing

Gaussian Blurring is the process of removing Gaussian noise from an image. This method takes the image, the kernel size and the standard deviation as arguments.

 img = cv2.imread('keiron.jpg', 1)
 img = cv2.resize(img, (320,210))
 # Apply Gaussian blur
 img1 = cv2.GaussianBlur(img,(5,5),2)
 # display the images
 cv2.imshow('Original', img)
 cv2.imshow('Gaussian', img1)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
Gaussian blur

Median blur is known for its salt-and-pepper noise removal. Unwanted small white and black dots on an image is removed with this tool.

 img = cv2.imread('tree.png', 0)
 # Apply median blur
 img1 = cv2.medianBlur(img,3)
 # display the images
 cv2.imshow('Original', img)
 cv2.imshow('Median', img1)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
Median blur

Bilateral Filter is used when there is a need for both noise filtering and edges retention. This method detects sharp edges and keeps it as such without any blur.

 img = cv2.imread('keiron.jpg', 1)
 img = cv2.resize(img, (320,210))
 # Apply Bilateral Filter
 img1 = cv2.bilateralFilter(img,7,100,100)
 # display the images
 cv2.imshow('Original', img)
 cv2.imshow('Bilateral', img1)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
Bilateral filter processing

Image Gradients 

Image gradients are the rate of change of pixel values in either x or y direction or in both x and y directions. It helps identify the sudden changes in pixel values. In other words, it helps detect the edges. If the gradient is applied in x-direction, vertical edges are detected. If the gradient is applied in y-direction, horizontal edges are detected.

In OpenCV, Laplacian gradient detects edges both horizontally and vertically, while Sobel gradient detects edges either horizontally or vertically.

 import numpy as np
 img = cv2.imread('chessboard.jpg', 0)
 img = cv2.resize(img, (300,200))
 # Laplacian image gradient
 lap = np.uint8(np.absolute(cv2.Laplacian(img,cv2.CV_64F, ksize=1)))
 # display the images
 cv2.imshow('Original', img)
 cv2.imshow('Lpalacian', lap)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
Laplacian gradient
 img = cv2.imread('chessboard.jpg', 0)
 img = cv2.resize(img, (300,200))
 # Sobel image gradient
 vertical = np.uint8(np.absolute(cv2.Sobel(img,cv2.CV_64F, 1,0, ksize=1)))
 horizon = np.uint8(np.absolute(cv2.Sobel(img,cv2.CV_64F, 0,1, ksize=1)))
 # display the images
 cv2.imshow('Vertical', vertical)
 cv2.imshow('Horizontal', horizon)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
Sobel gradients

Vertical and horizontal edges can be blended together to generate an image gradient with all edges.

 Sobel = cv2.bitwise_or(vertical, horizon)
 cv2.imshow('Sobel', Sobel)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
Sobel gradients on image

Canny Edge Detection

Canny Edge Detection is a powerful edge detection algorithm that performs Gaussian noise filtering, Sobel based horizontal and vertical edges detection, non-maximum suppression to remove unwanted edge points and hysteresis thresholding with two limiting thresholds to have thin and strong edges.

 img = cv2.imread('chessboard.jpg', 0)
 img = cv2.resize(img, (450,300))
 def null(x):
     pass
 # create trackbars to control threshold values
 cv2.namedWindow('Canny')
 cv2.resizeWindow('Canny', (450,300))
 cv2.createTrackbar('MIN', 'Canny', 80,255, null)
 cv2.createTrackbar('MAX', 'Canny', 120,255, null)
 while True:
     # get Trackbar position
     a = cv2.getTrackbarPos('MIN', 'Canny')
     b = cv2.getTrackbarPos('MAX', 'Canny')
     # Canny Edge detection
     # arguments: image, min_val, max_val
     canny = cv2.Canny(img,a,b)
     # display the images
     cv2.imshow('Canny', canny)
     k = cv2.waitKey(1) & 0xFF
     if k == ord('q'):
         break
 cv2.destroyAllWindows() 
Canny Edge detection

By changing the minimum and maximum threshold values, we can improve the results.

Image Contours

Image contours are the continuous shape outlines present in an image. OpenCV detects the contours present in an image and collects its coordinates as a list. The collected contours can be drawn over the original image back.

 img = cv2.imread('valerie.jpg', 1)
 img = cv2.resize(img, (320,480))
 # show original image
 cv2.imshow('Original', img)
 # binary thresholding
 gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
 val,th = cv2.threshold(gray, 127,255,cv2.THRESH_BINARY)
 # find contours
 contours,_ = cv2.findContours(th, 
                               cv2.RETR_TREE,
                               cv2.CHAIN_APPROX_NONE)
 # draw contours on original image
 # arguments: image, contours list, index of contour, colour, thickness
 cv2.drawContours(img, contours, -1, (0,0,255),1)
 cv2.imshow('Contour', img)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
image contours

Contours are the fundamental building blocks for object shape detection, motion detection and image segmentation. Contours are collected in a Python list. We can draw a particular contour or contours by indexing or slicing those contours alone.

 # slice the contours list
 face = contours[455:465]
 cv2.drawContours(img, face, -1, (0,0,255),1)
 cv2.imshow('Contour', img)
 cv2.waitKey(0)
 cv2.destroyAllWindows() 
face image contours

Wrapping Up

In this tutorial, we discussed how image processing is performed with OpenCV in Python. We studied the following methods with hands-on codes and visualizations:

  1. Simple Thresholding on Images
  2. Adaptive Thresholding on Images
  3. Image Smoothing
  4. Image Gradients
  5. Canny Edge Detection
  6. Image Contours

With a basic understanding of image processing, users can dive deeper into real-world Computer Vision problems.

Note: The open-source images used in this article are licensed to reuse without any permission.

Further Reading:

  1. Getting Started with OpenCV in Python
  2. Real-time GUI Interactions with OpenCV in Python
  3. Official documentation of OpenCV
  4. Credit Card Reader using OpenCV
  5. Face Swapping Application using OpenCV
  6. Face Filter using OpenCV

Download our Mobile App

Rajkumar Lakshmanamoorthy
A geek in Machine Learning with a Master's degree in Engineering and a passion for writing and exploring new things. Loves reading novels, cooking, practicing martial arts, and occasionally writing novels and poems.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

15th June | Online

Building LLM powered applications using LangChain

17th June | Online

Mastering LangChain: A Hands-on Workshop for Building Generative AI Applications

Jun 23, 2023 | Bangalore

MachineCon 2023 India

26th June | Online

Accelerating inference for every workload with TensorRT

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox
MOST POPULAR