How To Create A Virtual Painting App Using OpenCV?

Computer vision is a science of teaching computers to see. With the state of the art algorithms, this technology is behind many applications like self-driving cars, image recognition, medical diagnosis etc. The best part of computer vision is these techniques are used for detecting cancerous cells which helps in saving lives by adding filters to your face to entertain you. Similarly, it has a wide variety of interesting and useful applications. In this article, we will do hands-on implementation on how to create a virtual painting app without touching the keyboard, and just by drawing on-air that will be displayed on the screen.

Steps for this implementation:-

1. Importing libraries, assigning BGR values for color.
2. Creating a window that provides various options.
3. Capturing a video using a webcam and grabbing frames.
4. Using different morphing techniques 
5. Displaying output

So, now let’s begin with this interesting implementation.

Step-1 –  Importing required libraries 

import numpy as np

import cv2

from collections import deque

Using the below code snippet, we will declare the BGR values of the red color, which detects the red color and helps us to draw. 

Lower_red = np.array([161, 155, 84])

Upper_red = np.array([179, 255, 255])

Here i used red colour lower and upper ranges, you can try with your own color by setting BGR values

Using the below code snippet, we will use a deque function, which is used to store separate colors that helps in creating buttons on the window.

bpoints = [deque(maxlen=512)]

gpoints = [deque(maxlen=512)]

rpoints = [deque(maxlen=512)]

ypoints = [deque(maxlen=512)]

bindex = 0,gindex = 0,rindex = 0,yindex = 0

colors = [(255, 0, 0), (0, 255, 0), (0, 0, 255), (0, 255, 255)]

colorIndex = 0

Step-2 – Creating a window, that provides various options.

Using the below code snippet, we will create a window that displays the different color options to draw and also it creates the buttons using cv2.putText. 

Window = np.zeros((471,636,3)) + 255

Window = cv2.rectangle(paintWindow, (40,1), (140,65), (0,0,0), 2)

Window = cv2.rectangle(paintWindow, (160,1), (255,65), colors[0], -1)

Window = cv2.rectangle(paintWindow, (275,1), (370,65), colors[1], -1)

Window = cv2.rectangle(paintWindow, (390,1), (485,65), colors[2], -1)

Window = cv2.rectangle(paintWindow, (505,1), (600,65), colors[3], -1)

cv2.putText(Window, "CLEAR ALL", (49, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 0), 2, cv2.LINE_AA)

cv2.putText(Window, "BLUE", (185, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)

cv2.putText(Window, "GREEN", (298, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)

cv2.putText(Window, "RED", (420, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)

cv2.putText(Window, "YELLOW", (520, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (150,150,150), 2, cv2.LINE_AA)

cv2.namedWindow('Paint', cv2.WINDOW_AUTOSIZE)

As you can see in the output we created a normal window now we need to set the webcam and place this template on the webcam video.

Step-3 – Capturing a video using a webcam and grabbing frames.

Here we are capturing the video, using a webcam so we set videoCapture to zero.

camera = cv2.VideoCapture(0)

Using the below code snippet, we will run a loop to grab the frames from the webcam and to display the dynamic results.

while True:

    (grabbed, frame) = camera.read()

    frame = cv2.flip(frame, 1)

    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

if not grabbed:

        Break

Step-4 – Using different morphing techniques

Using the below code snippet, we will preprocess using the input( the text drawn on-air) using morphing techniques erode, morphologyEx, dilate.

    kernel = np.ones((5, 5), np.uint8) 

    Mask = cv2.inRange(hsv, Lower_red, Upper_red)

    Mask = cv2.erode(Mask, kernel, iterations=2)

    Mask = cv2.morphologyEx(Mask, cv2.MORPH_OPEN, kernel)

    Mask = cv2.dilate(Mask, kernel, iterations=1)

Using the below code snippets, we will use counters, it holds the data in an unordered collection it allows you to count the items in the iterable list. 

(cnts, _) = cv2.findContours(blueMask.copy(), cv2.RETR_EXTERNAL,

      cv2.CHAIN_APPROX_SIMPLE)

    center = None

Using the below code snippet, when it detects the red color and around that color, we create a bounding circle.

if len(cnts) > 0:

        cnt = sorted(cnts, key = cv2.contourArea, reverse = True)[0]

        ((x, y), radius) = cv2.minEnclosingCircle(cnt)

        cv2.circle(frame, (int(x), int(y)), int(radius), (0, 255, 255), 2)

        M = cv2.moments(cnt)

        center = (int(M['m10'] / M['m00']), int(M['m01'] / M['m00'])

In the below code snippet, if counters are stored then it runs the loop. using the deque function.

        if center[1] <= 65:

            if 40 <= center[0] <= 140: # Clear All

                bpoints = [deque(maxlen=512)]

                gpoints = [deque(maxlen=512)]

                rpoints = [deque(maxlen=512)]

                ypoints = [deque(maxlen=512)]

                bindex = 0, gindex = 0, rindex = 0,yindex = 0

                paintWindow[67:,:,:] = 255

            elif 160 <= center[0] <= 255:

                    colorIndex = 0

            elif 275 <= center[0] <= 370:

                    colorIndex = 1

            elif 390 <= center[0] <= 485:

                    colorIndex = 2

            elif 505 <= center[0] <= 600:

                    colorIndex = 3

        else :

            if colorIndex == 0:

                bpoints[bindex].appendleft(center)

            elif colorIndex == 1:

                gpoints[gindex].appendleft(center)

            elif colorIndex == 2:

                rpoints[rindex].appendleft(center)

            elif colorIndex == 3:

                ypoints[yindex].appendleft(center)

Step-5 – Displaying output

 In the below code snippet, we are implementing the draw option by taking our designed window and applying it to the webcam video. 

 points = [bpoints, gpoints, rpoints, ypoints]

    for i in range(len(points)):

        for j in range(len(points[i])):

            for k in range(1, len(points[i][j])):

                if points[i][j][k - 1] is None or points[i][j][k] is None:

                    continue

                cv2.line(frame, points[i][j][k - 1], points[i][j][k], colors[i], 2)

                cv2.line(paintWindow, points[i][j][k - 1], points[i][j][k], colors[i], 2)

In the below code snippet, we are displaying the output video

   cv2.imshow("Web_cam_video", frame)

  cv2.imshow("Paint", paintWindow) 

Conclusion

In the above implementation, we demonstrated how to create a virtual paint application using OpenCV tool. We had given lower and upper ranges of red color so we can use red color as a pencil to draw virtually and it reflects on your system screen. Although my drawings are bad,  it’s fun to implement.