Now Reading
Guide To Kornia: An OpenCV-inspired PyTorch Framework

Guide To Kornia: An OpenCV-inspired PyTorch Framework

Kornia

Kornia is an open-source Python library inspired by OpenCV designed to handle generic Computer Vision tasks. It was introduced by Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee and Gary Bradski in October, 2019 (research paper). 

Kornia leverages PyTorch library at its backend in terms of model’s efficiency and reverse-mode auto-differentiation for defining and computing complex functions’ gradients. It comprises a subset of packages having operators that act as an input to neural networks for performing a wide range of tasks such as image transformations, depth estimation, epipolar geometry, filtering and edge-detection applicable on high-dimensional tensors. Unlike conventional CPU-based CV libraries such as torchvision and scikit-image, several standard deep learning functions can be implemented on GPUs using Kornia. 

Kornia bridges the gap between two simple yet powerful libraries namely, OpenCV and PyTorch. Though solely based on traditional CV solutions like torchvision, tf.image, PIL and skimage, it enables differentiable programming for CV applications by utilizing the crucial properties of PyTorch like GPU hardware acceleration, differentiability and distributed data-flows.

The following table compares Kornia with some important CV libraries/modules:

Library/ModuleCPU supportedGPU supportedBatch ProcessingDifferentia-bleDistributedSupports multidimensional array
torchvisionYesNoNoNo NoNo
scikit-imageYesNoNoNoNoNo
opencvYesYesNoNoNoNo
tf.imageYesYesYesYesYesYes
KorniaYesYesYesYesYesYes

Source of above summary: Research paper

Components of the library

Practical implementation

Here, we demonstrate three use cases of Kornia – blurring a custom image, changing its color space, and adjusting its colors. The code is test on Google colab with Python 3.7.10 and Kornia 0.4.1 versions.

We have used the following image for demonstration:

Kornia input image

Image source

Step-wise explanation of the code is as follows:

First, install the library using pip command.

!pip install kornia

Import required libraries and modules

 import torch
 import torchvision
 import kornia
 import cv2
 import numpy as np
 import matplotlib.pyplot as plt 

Image blurring using Kornia

  1. Read the input image 
 #Read the image using OpenCV and convert it to a numpy array
 input_image: np.ndarray = cv2.imread('img2.jpg')
 #Convert the image color space from BGR to RGB
 input_image = cv2.cvtColor(input_image, cv2.COLOR_BGR2RGB) 
  1. Convert the image to a torch 4D tensor 
tensor_image: torch.tensor = kornia.image_to_tensor(input_image, keepdim=False)
  1. Create an operator that blurs the tensor image using Gaussian filter 

gaussian = kornia.filters.GaussianBlur2d((11, 11), (10.5, 10.5))

Where, (11,11) is the size of the kernel and (1.05,10.5) is the standard deviation of the kernel

  1. Convert the tensor image to float type and apply the gaussian operator defined in previous step to blur the image

blur_image: torch.tensor = gaussian(tensor_image.float())

  1. Convert the blurred tensor image back to a numpy array

final_blur_image: np.ndarray = kornia.tensor_to_image(blur_image.byte())

  1. Plot the input and output images
 fig, ax = plt.subplots(1, 2, figsize=(16, 10))
 ax = ax.ravel() #flatten the axis to a 1D array
 #Original image
 ax[0].axis('off')  #turn off the axis lines and labels
 ax[0].set_title('Original image')  #Title of image
 ax[0].imshow(input_image)   #Display the image
 #Blurred image
 ax[1].axis('off')    #turn off the axis lines and labels
 ax[1].set_title('Blurred image')   #Title of image
 ax[1].imshow(final_blur_image)  #Display the image 

Output:

Kornia output1

Color space conversion using Kornia

  1. Read the input colored image and convert it to a numpy array

bgr_image: np.ndarray = cv2.imread('img2.jpg', cv2.IMREAD_COLOR)

  1. Convert the image to a torch tensor

tensor_bgr: torch.Tensor = kornia.image_to_tensor(bgr_image, keepdim=False)

  1. Define functions to flip the image horizontally and vertically and rotate it by 180 degrees.
 def horizontal_flip(input: torch.Tensor) -> torch.Tensor:
     return torch.flip(input, [-1])
 #torch.flip() reverses the order of the input tensor image along given axis 
 def vertical_flip(input: torch.Tensor) -> torch.Tensor:
     return torch.flip(input, [-2])
 def rotate_180(input: torch.Tensor) -> torch.Tensor:
     return torch.flip(input, [-2, -1]) 
  1. Define a function to plot a batch of images
 def imshow(input: torch.Tensor):
 #Create a grid with 2 rows
 output: torch.Tensor = torchvision.utils.make_grid(input, 
 nrow=2, padding=5)
 #Convert the grid of images to numpy ndarray
       output_np: np.ndarray = kornia.tensor_to_image(output)
       plt.imshow(output_np)  #Plot the grid
       plt.axis('off')  #Turn off the axis lines and labels
       plt.show()   #Display the grid 
  1. Create a batch of images from BGR image
#Apply the functions defined in step (3) to the input tensor image and then #concatenate resulting tensors
 batch_bgr = torch.cat([tensor_bgr, horizontal_flip(tensor_bgr), vertical_flip(tensor_bgr), rotate_180(tensor_bgr)])
 #Display the batch of images
 imshow(batch_bgr)   

Output:

output2
  1. BGR to RGB color space conversion
 batch_rgb = kornia.bgr_to_rgb(batch_bgr)  #conversion
 imshow(batch_rgb) #display the resulting  

Output:

output3
  1. RGB to Grayscale color space conversion
 batch_gray = kornia.rgb_to_grayscale(batch_rgb.float() / 255.) 
 imshow(batch_gray)  #display the grayscale image 

Output:

output4
  1. RGB to HSV (Hue, Saturation, Value) color space conversion
 batch_hsv = kornia.rgb_to_hsv(batch_rgb.float() / 255.)
 imshow(batch_hsv[:, 2:3])  #display HSV image 

Output:

Kornia output5

Color adjustment

  1. Read the input image in BGR format

bgr_image: np.ndarray = cv2.imread('img2.jpg', cv2.IMREAD_COLOR)

  1. Convert input image to torch tensor

tensor_bgr: torch.Tensor = kornia.image_to_tensor(bgr_image)

Convert the image from BGR to RGB

See Also
Own Data For Object Detection

tensor_rgb: torch.Tensor = kornia.bgr_to_rgb(tensor_bgr)

  1. Expand the image 
 tensor_rgb = tensor_rgb.expand(4, -1, -1, -1)  
 # 4 x Channels x Height x Width
 tensor_rgb = tensor_rgb.float() / 255. #Normalize the expanded image 

Define a function to create a batch of images

 def imshow(input: torch.Tensor):
 #create grid having 2 rows of images
     output: torch.Tensor = torchvision.utils.make_grid(input, nrow=2, 
     padding=5)
 #Convert tensor images to numpy ndarray
     output_np: np.ndarray = kornia.tensor_to_image(output)
 #Plot the grid of images
     plt.imshow(output_np)  
     plt.axis('off')
     plt.show() 

Display the batch of RGB images

imshow(tensor_rgb)

Output:

Kornia output7
  1. Adjust brightness of the batch images
 tensor_brightness: torch.Tensor = kornia.adjust_brightness(tensor_rgb, 0.6)
 #Where, 0.6 is the factor to adjust brightness of each element in the batch
 imshow(tensor_brightness)  #display the resulting batch 

Output:

Kornia output8
  1. Adjust contrast of the batch images
 tensor_contrast: torch.Tensor = kornia.adjust_contrast(tensor_rgb, 0.2)
 #0.2 is the contrast adjustment factor per batch element
 imshow(tensor_contrast)  #dispaly the resulting batch 

Output:

Kornia output9
  1. Adjust gamma correction of the images
 tensor_gamma: torch.Tensor = kornia.adjust_gamma(tensor_rgb, gamma=3., gain=1.5)
 #where, ‘gamma’ is a non-negative number and ‘gain’ is the constant multiplier
 imshow(tensor_gamma)  #display resulting batch 

Output:

Kornia output10
  1. Adjust saturation level of the batch images
 tensor_saturated: torch.Tensor = kornia.adjust_saturation(tensor_rgb, 0.2)
 #where 0.2 is the saturation factor
 imshow(tensor_saturated)  #display the resulting batch 

Output:

Kornia output11
  1. Adjust hue of the images
 tensor_hue: torch.Tensor = kornia.adjust_hue(tensor_rgb, 0.5)
 #where 0.5 gives measure of how much to shift the hue channel
 imshow(tensor_hue)  #display the resulting batch 

Output:

Kornia output12
  • Google colab notebook of the above-explained implementation is available here.

References

To get in-depth understanding of the Kornia library, refer to the following web links:

What Do You Think?

Join Our Telegram Group. Be part of an engaging online community. Join Here.

Subscribe to our Newsletter

Get the latest updates and relevant offers by sharing your email.

Copyright Analytics India Magazine Pvt Ltd

Scroll To Top