Advertisement

Active Hackathon

How To Create A Virtual Painting App Using OpenCV?

virtual painting

Computer vision is a science of teaching computers to see. With the state of the art algorithms, this technology is behind many applications like self-driving cars, image recognition, medical diagnosis etc. The best part of computer vision is these techniques are used for detecting cancerous cells which helps in saving lives by adding filters to your face to entertain you. Similarly, it has a wide variety of interesting and useful applications. In this article, we will do hands-on implementation on how to create a virtual painting app without touching the keyboard, and just by drawing on-air that will be displayed on the screen.

Steps for this implementation:-

  1. Importing libraries, assigning BGR values for color.
  2. Creating a window that provides various options.
  3. Capturing a video using a webcam and grabbing frames.
  4. Using different morphing techniques 
  5. Displaying output

So, now let’s begin with this interesting implementation.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

Step-1 –  Importing required libraries 

import numpy as np

import cv2

from collections import deque

Using the below code snippet, we will declare the BGR values of the red color, which detects the red color and helps us to draw. 

Lower_red = np.array([161, 155, 84])

Upper_red = np.array([179, 255, 255])

Here i used red colour lower and upper ranges, you can try with your own color by setting BGR values

Using the below code snippet, we will use a deque function, which is used to store separate colors that helps in creating buttons on the window.

bpoints = [deque(maxlen=512)]

gpoints = [deque(maxlen=512)]

rpoints = [deque(maxlen=512)]

ypoints = [deque(maxlen=512)]

bindex = 0,gindex = 0,rindex = 0,yindex = 0

colors = [(255, 0, 0), (0, 255, 0), (0, 0, 255), (0, 255, 255)]

colorIndex = 0

Step-2 – Creating a window, that provides various options.

Using the below code snippet, we will create a window that displays the different color options to draw and also it creates the buttons using cv2.putText. 

Window = np.zeros((471,636,3)) + 255

Window = cv2.rectangle(paintWindow, (40,1), (140,65), (0,0,0), 2)

Window = cv2.rectangle(paintWindow, (160,1), (255,65), colors[0], -1)

Window = cv2.rectangle(paintWindow, (275,1), (370,65), colors[1], -1)

Window = cv2.rectangle(paintWindow, (390,1), (485,65), colors[2], -1)

Window = cv2.rectangle(paintWindow, (505,1), (600,65), colors[3], -1)

cv2.putText(Window, "CLEAR ALL", (49, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 0), 2, cv2.LINE_AA)

cv2.putText(Window, "BLUE", (185, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)

cv2.putText(Window, "GREEN", (298, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)

cv2.putText(Window, "RED", (420, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2, cv2.LINE_AA)

cv2.putText(Window, "YELLOW", (520, 33), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (150,150,150), 2, cv2.LINE_AA)

cv2.namedWindow('Paint', cv2.WINDOW_AUTOSIZE)

As you can see in the output we created a normal window now we need to set the webcam and place this template on the webcam video.

Step-3 – Capturing a video using a webcam and grabbing frames.

Here we are capturing the video, using a webcam so we set videoCapture to zero.

camera = cv2.VideoCapture(0)

Using the below code snippet, we will run a loop to grab the frames from the webcam and to display the dynamic results.

while True:

    (grabbed, frame) = camera.read()

    frame = cv2.flip(frame, 1)

    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

if not grabbed:

        Break

Step-4 – Using different morphing techniques

Using the below code snippet, we will preprocess using the input( the text drawn on-air) using morphing techniques erode, morphologyEx, dilate.

    kernel = np.ones((5, 5), np.uint8) 

    Mask = cv2.inRange(hsv, Lower_red, Upper_red)

    Mask = cv2.erode(Mask, kernel, iterations=2)

    Mask = cv2.morphologyEx(Mask, cv2.MORPH_OPEN, kernel)

    Mask = cv2.dilate(Mask, kernel, iterations=1)

Using the below code snippets, we will use counters, it holds the data in an unordered collection it allows you to count the items in the iterable list. 

(cnts, _) = cv2.findContours(blueMask.copy(), cv2.RETR_EXTERNAL,

      cv2.CHAIN_APPROX_SIMPLE)

    center = None

Using the below code snippet, when it detects the red color and around that color, we create a bounding circle.

if len(cnts) > 0:

        cnt = sorted(cnts, key = cv2.contourArea, reverse = True)[0]

        ((x, y), radius) = cv2.minEnclosingCircle(cnt)

        cv2.circle(frame, (int(x), int(y)), int(radius), (0, 255, 255), 2)

        M = cv2.moments(cnt)

        center = (int(M['m10'] / M['m00']), int(M['m01'] / M['m00'])

In the below code snippet, if counters are stored then it runs the loop. using the deque function.

        if center[1] <= 65:

            if 40 <= center[0] <= 140: # Clear All

                bpoints = [deque(maxlen=512)]

                gpoints = [deque(maxlen=512)]

                rpoints = [deque(maxlen=512)]

                ypoints = [deque(maxlen=512)]

                bindex = 0, gindex = 0, rindex = 0,yindex = 0

                paintWindow[67:,:,:] = 255

            elif 160 <= center[0] <= 255:

                    colorIndex = 0

            elif 275 <= center[0] <= 370:

                    colorIndex = 1

            elif 390 <= center[0] <= 485:

                    colorIndex = 2

            elif 505 <= center[0] <= 600:

                    colorIndex = 3

        else :

            if colorIndex == 0:

                bpoints[bindex].appendleft(center)

            elif colorIndex == 1:

                gpoints[gindex].appendleft(center)

            elif colorIndex == 2:

                rpoints[rindex].appendleft(center)

            elif colorIndex == 3:

                ypoints[yindex].appendleft(center)

Step-5 – Displaying output

 In the below code snippet, we are implementing the draw option by taking our designed window and applying it to the webcam video. 

 points = [bpoints, gpoints, rpoints, ypoints]

    for i in range(len(points)):

        for j in range(len(points[i])):

            for k in range(1, len(points[i][j])):

                if points[i][j][k - 1] is None or points[i][j][k] is None:

                    continue

                cv2.line(frame, points[i][j][k - 1], points[i][j][k], colors[i], 2)

                cv2.line(paintWindow, points[i][j][k - 1], points[i][j][k], colors[i], 2)

In the below code snippet, we are displaying the output video

   cv2.imshow("Web_cam_video", frame)

  cv2.imshow("Paint", paintWindow) 

Output video. Although my drawings are bad,  it’s fun to implement.

Conclusion

In the above implementation, we demonstrated how to create a virtual paint application using OpenCV tool. We had given lower and upper ranges of red color so we can use red color as a pencil to draw virtually and it reflects on your system screen. Although my drawings are bad,  it’s fun to implement.

More Great AIM Stories

Prudhvi varma
AI enthusiast, Currently working with Analytics India Magazine. I have experience of working with Machine learning, Deep learning real-time problems, Neural networks, structuring and machine learning projects. I am a Computer Vision researcher and I am Interested in solving real-time computer vision problems.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MOST POPULAR

Data Science Skills Survey 2022 – By AIM and Great Learning

Data science and its applications are becoming more common in a rapidly digitising world. This report presents a comprehensive view to all the stakeholders — students, professionals, recruiters, and others — about the different key data science tools or skillsets required to start or advance a career in the data science industry.

How to Kill Google Play Monopoly

The only way to break Google’s monopoly is to have localised app stores with an interface as robust as Google’s – and this isn’t an easy ask. What are the options?

[class^="wpforms-"]
[class^="wpforms-"]