7 Popular AI Projects On Gesture Gaming

AI has made several breakthroughs when it comes to implementation into games. The functionalities of AI in video games include various domains such as real-time facial emotion recognition, automated difficulty adaptation, sentiment analysis, non-verbal bodily motion, lip-synchronised speech and more.

This technique has been used in games to enhance graphical realism, to generate levels, sceneries and storylines, to establish player profiles, balance complexity or to add intelligent behaviours to non-playing characters. 

In this article, we list down the seven popular AI projects that work on gesture gaming.

(The list is no particular order). 

1| Playing Snake Game Using Hand Gestures

About: This project will help you understand how to use TensorFlow object detection API with the computer’s webcam to play a snake game by using hand gestures. The steps in this project include generating dataset, converting trained and test datasets into tfrecord format, training a pre-trained model using generated data, integrate the trained model with the snake game and other such. 

Know more here.

2| SkyFall: Gesture Controlled Web-based Game using Tensorflow Object Detection API

About: Skyfall is a physics-based game in which users can control an onscreen paddle simply by moving their hands in front of a webcam. In this project, light-weight Convolutional Neural Networks, such as MobileNet, SSD are used to detect the users’ hands which are then mapped to the controls of the game. You will understand how the integration of a well-trained and lightweight hand detection model is used to robustly track player hands and enable “body as an input” interaction in real-time.

Know more here.

3| Gesture-Controlled Space Battle Game in PictoBlox using Human Body Detection

About: In this project, you will learn how to make a gesture-controlled space battle game in PictoBlox and control it using the tip of your fingers. You will get a ride through building this project in PictoBlox, which is the scratch-based graphical programming software. It will also enable writing the script for the Space Battle Game and more.  

Know more here.

4| AI Gesture-Controlled 2-Wheel Drive Robot

About: In this project, you will learn how to control a 2-Wheel drive robot with hand gestures using PictoBlox AI’s machine learning extension. The building guide includes pre-requisites to make gesture-controlled robots, how to train the machine learning model to recognise the gestures, writing the script and more. 

Know more here.

5| Hand Gesture Recognition using Deep Learning in Matlab

About: This project uses deep learning techniques for image recognition systems to find the hand gesture. The model is trained with static hand gesture images, and the convolutional neural network is created without using a pre-trained model. 

Know more here.

6| HeroMirror Interactive: A Gesture Controlled Augmented Reality Gaming Experience

About: In this project, a deep learning-based gesture recognition method allows the detection of predefined gestures using a regular camera. The captured frames are processed by a state-of-the-art convolutional neural network trained for object detection. The researchers proposed a monocular camera-based gesture recognition method that can ensure a similar level of recognition accuracy as depth-camera based solutions. Furthermore, they also built an augmented reality gaming experience that is built on the top of the proposed method.

Know more here.

7| Recognition-Based Gesture Spotting In Video Games

About: In this project, the researchers proposed a gesture spotting method that combines gesture recognition. It recognises the meaningful movements while concurrently separating unintentional movements from a given image sequence. The researchers then applied this method to recognise the upper-body human gestures as an interface between a video game called Quake II and its user.  

Know more here.

More Great AIM Stories

Ambika Choudhury
A Technical Journalist who loves writing about Machine Learning and Artificial Intelligence. A lover of music, writing and learning something out of the box.

More Stories


8th April | In-person Conference | Hotel Radisson Blue, Bangalore

Organized by Analytics India Magazine

View Event >>

30th Apr | Virtual conference

Organized by Analytics India Magazine

View Event >>

Vijaysinh Lendave
How to do Pose Estimation With MoveNet

MoveNet is an ultra-fast and accurate estimator which detects the 17 key points of a body part, as shown above. This model is hosted on Tensorflow-Hub along with its two variants called as lighting and thunder. Lightning is used where critical latency application is hosted, while thunder variant where high accuracy requires. Both the estimators run for more than 30 FPS on most modern machines and mobile phones.

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM