How I Played The Chrome-Dino Game With Just My Palm

What do we see on Chrome when there is no internet connection? The classic Chrome Dinosaur Game: T-Rex Runner Game (aka the No Internet Game)! Simply press the space bar and the dinosaur will start running. Press the arrow-up key to make the dinosaur jump; the longer you press the key, the higher it jumps. Every single one of us must’ve played this game at least once. Has it ever crossed your mind that you can play this very game using just your palm? To me, it has! 

Here’s how I played the Chrome-Dino game with just my palm! Is it possible, you may ask? It is very much possible with a special Python module named PyAutoGUI.

What libraries did I use?

  1. NumPy (a Python library)
  2. OpenCV (another Python library, focussing on real-time applications) and of course,
  3. PyAutoGUI 

import numpy as np
import cv2
import math
import pyautogui


Sign up for your weekly dose of what's up in emerging technology.

How did I do it?

  • Open up the camera and draw a rectangle
  • Blur the image and convert it into HSV
  • Erode, Dilate and Threshold
  • Contour 
  • Run the code and play!

Do you want to implement the same? Follow these steps below!

Step-1: Open the camera to read input and draw a rectangle on the frame. I used a built-in function of OpenCV to draw the rectangle. Also, read the hand image from the frame. Feel free to change the frame size!

Download our Mobile App

cap = cv2.VideoCapture(0)
while True:
    ret, frame =

    # collect hand gestures
    cv2.rectangle(frame, (100, 100), (300, 300), (255, 0, 0), 0)
    hand_img = frame[100:300, 100:300]

Step-2: Smoothen the input image using Gaussian Blur. Image smoothing techniques help in reducing the noise. Gaussian blur is used to reduce the amount of noise and remove dark spots (speckles) within the image. 

HSV (Hue, Saturation and Lightness) separates the image intensity from the colour information. Hence, I preferred HSV over BGR and converted the initial BGR image into HSV. 

blur = cv2.GaussianBlur(hand_img, (3, 3), 0)
hsv = cv2.cvtColor(blur, cv2.COLOR_BGR2HSV)

#binary image with where white will be skin colors and rest is black
skin = cv2.inRange(hsv, np.array([2, 0, 0]), np.array([20, 255, 255]))
kernel = np.ones((5, 5))

Step-3: Erosion, dilation and thresholding. 

Erosion helps in eroding away the boundaries of foreground object. Dilation is the exact opposite of osion. It helps in increasing the foreground object (increases the white region in the image). Thresholding is a method of image segmentation. It is used to create binary objects. If pixel value is greater than the threshold value, it is assigned one value (white), otherwise it is assigned another value (black). Take note that the larger the kernel value is, the more blurred the image will be. 

dilation = cv2.dilate(skin, kernel, iterations=1)
erosion = cv2.erode(dilation, kernel, iterations=1)
filtered = cv2.GaussianBlur(erosion, (3, 3), 0)
ret, thresh = cv2.threshold(filtered, 127, 255, 0)
contours, hierachy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

Step-4: Detecting contours. 

Wherever a white (skin colour) pixel is detected, an outline will be drawn. To detect the outline of my palm, I used cv2.moments and found out the centroid of the contour. When your palm is detected completely, space bar will be pressed so the T-Rex can jump. When your palm is not detected, the T-Rex will not jump (it just runs). This is done with the help of PyAutoGUI. 

PyAutoGUI is a cross-platform GUI automation Python module for human beings. Using PyAutoGUI, we can control the mouse and the keyboard just by writing a few lines of code. 

        contour = max(contours, key=lambda x: cv2.contourArea(x))
        x, y, w, h = cv2.boundingRect(contour)
  cv2.rectangle(hand_img, (x, y), (x + w, y + h), (0, 0, 255), 0)

hull = cv2.convexHull(contour)
draw = np.zeros(hand_img.shape, np.uint8)
cv2.drawContours(draw, [contour], -1, (0, 255, 0), 0)
cv2.drawContours(draw, [hull], -1, (0, 0, 255), 0)
hull = cv2.convexHull(contour, returnPoints=False)
defects = cv2.convexityDefects(contour, hull)
count_defects = 0
for i in range(defects.shape[0]):
            s, e, f, d = defects[i, 0]
            start = tuple(contour[s][0])
            end = tuple(contour[e][0])
            far = tuple(contour[f][0])
            a = math.sqrt((end[0] – start[0]) ** 2 + (end[1] – start[1]) ** 2)
            b = math.sqrt((far[0] – start[0]) ** 2 + (far[1] – start[1]) ** 2)
            c = math.sqrt((end[0] – far[0]) ** 2 + (end[1] – far[1]) ** 2)
            angle = (math.acos((b ** 2 + c ** 2 – a ** 2) / (2 * b * c)) * 180) / 3.14
            if angle <= 90:
                count_defects += 1
      , far, 1, [0, 0, 255], -1)
            cv2.line(hand_img, start, end, [0, 255, 0], 2)
        # if the codition matches, press space
        if count_defects >= 4:
            cv2.putText(frame, “JUMP”, (115, 80), cv2.FONT_HERSHEY_SIMPLEX, 2, 2, 2)

Step-5: Exit the capture window by pressing the key ‘q’. 

cv2.imshow(“Gesture”, frame)
    if cv2.waitKey(1) == ord(‘q’):

Step-6: Finally, run the program (it might take a while depending on the speed of your computer). Make sure you keep your hand in the rectangle that is visible on the screen. You may change the dimensions of it according to your convenience. 

And that is how I played a game with just my palm! For best results, use this in a brightly lit area. You can use the online version if you don’t want to switch off your internet connection every time you want to play the game. or

This mini project is well suited for someone who is just getting started with OpenCV and Image Processing.

For the complete code, head to my GitHub profile, link to which is given below!

Or my LinkedIn profile:

Support independent technology journalism

Get exclusive, premium content, ads-free experience & more

Rs. 299/month

Subscribe now for a 7-day free trial

More Great AIM Stories

Srinija Dharani
I’m a Computer Vision geek and I love to code! Alongside pursuing Computer Science Engineering at GITAM University Bangalore, I develop front-end websites for personal and professional uses. I also have deep interests in music and art.

AIM Upcoming Events

Early Bird Passes expire on 3rd Feb

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

All you need to know about Graph Embeddings

Embeddings can be the subgroups of a group, similarly, in graph theory embedding of a graph can be considered as a representation of a graph on a surface, where points of that surface are made up of vertices and arcs are made up of edges