Top AI-Based Tools & Techniques For Deepfake Detection

Picture this — you receive a video of a politician’s before-election speech from a friend. You see that the politician in question is saying something outrageous. The sheer ridiculousness of his speech makes you forward it in your own circle, only to realise much later that the video was manipulated. But by now, the video has been circulated wide and far, with almost no way to recall it back. Such hyper-realistic manipulated videos are called deepfakes. These videos leverage powerful techniques from machine learning and artificial intelligence to generate highly deceptive visual and audio content.

The repercussions of such deepfakes have been dangerous with compromised videos of public figures in circulation which threaten their reputation. Worse, it is anticipated that deepfakes may even play a large role in swaying elections of countries. Notably, Facebook, Twitter, and TikTok have already banned such deepfake content on their platform.

With such major consequences of deepfakes, there have been several efforts to create tools, albeit with varying degrees of success, which can help detect them. Here, we discuss a few of them:

Microsoft’s Video Authenticator Tool

Launched in September this year ahead of the US elections 2020, this video authenticator tool is created by Microsoft. It can analyse a still photo or video to provide a confidence score to detect whether or not the media is manipulated. It detects the blending boundary of the deepfake and subtle grayscale elements that are undetectable to the human eye. It also provides this confidence score in real-time. 

The tool has been created with a public dataset from Face Forensics++ and has been tested using the Deepfake Detection Challenge Dataset. Both these datasets are leading technologies for training and testing deepfake detection technologies.

Additionally, the tech giant also introduced a new technology that can find doctored content and assure readers of its authenticity. It contains two parts – the first component is integrated into Microsoft Azure which allows the content creator to digital hashes and certificates which remains in the part of its metadata, and the second component is which helps the reader verifies and matches these certificates and hashes to check authenticity.

Deepfake Detection Using Biological Signals

The researchers from Binghamton University and Intel created a tool that goes beyond just deepfake detection and recognises the deepfake model behind the compromised video. This tool looks for unique biological and generative noise signals ‘deepfake heartbeats’ left by deepfake model videos. These signals are detected from 32 different spots in a person’s face, called the Photoplethysmography (PPG) cells.

The architecture of this model is based on convolutional neural networks with VGG blocks. It uses OpenFace library in Python for face detection, OpenCV for image processing, and Keras for neural network implementations. Like Video Authenticator tool from Microsoft, the learning setting of FaceCatcher is based on the FaceForensics++ (FF) dataset. As per the researchers, this tool has 97.29% accuracy for fake video detection.

Deepfake Detection Using Phoneme-Viseme Mismatches

This model is invented by researchers from Stanford University and the University of California. This technique exploits the fact that visemes, which denote the dynamics of the mouth shape, are sometimes different or inconsistent with the spoken phoneme. For example, for saying words such as mama, baba, and papa, there might be a phoneme-viseme mismatch, which can be used in detecting even spatially small and temporally localised manipulations in deepfake videos. The team used lipsync deep fakes created using three synthesis techniques – Audio-to-Video (A2V), Text-to-Video for short utterances (T2V-S), and Text-to-Video for longer utterances (T2V-L).

The technique was used for both manual video authentication and automatic approaches. For A2V, T2V-S, and T2V-L, this model showed an accuracy of 96.0%, 97.8%, and 97.4% for manual authentication, and 93.4%, 97.0%, and 92.8% for automatic authentication, respectively.

Forensic Technique Using Facial Movements

This model tracks facial expression and movements of a single video provided as input. It extracts the presence and strength of specific action units. This detection model has a one-class support vector model (SVM) which distinguishes an individual from others as well as comedic impersonators from deepfake impersonators.

This model utilises OpenFace, an open-source facial behaviour analysis toolkit to extract facial and head movements in a video. The library provides 2D and 3D facial landmark positions, head pose, and facial action units for each frame.

Recurrent Convolutional Strategy

The Recurrent Convolutional Strategy uses recurrent convolutional models (RCM) for detecting face manipulation in videos. RCM are a class of deep learning models which effectively exploit temporal information from image streams across domains. This technique can detect deepfake, Face2Face, and FaceSwap tampered faces in video streams. It was tested on the FaceForensics++ dataset and showed an accuracy of up to 97%, a 4.55% improvement from its preceding techniques.

Wrapping Up

The technique for creating deepfakes keeps developing and advancing. The tools and techniques discussed in this listicle do not present complete accuracy and effectiveness. However, they are making strides in the right direction for a battle that is much bigger.

Download our Mobile App

Shraddha Goled
I am a technology journalist with AIM. I write stories focused on the AI landscape in India and around the world with a special interest in analysing its long term impact on individuals and societies. Reach out to me at

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

15th June | Bangalore

Future Ready | Lead the AI Era Summit

15th June | Online

Building LLM powered applications using LangChain

17th June | Online

Mastering LangChain: A Hands-on Workshop for Building Generative AI Applications

20th June | Bangalore

Women in Data Science (WiDS) by Intuit India

Jun 23, 2023 | Bangalore

MachineCon 2023 India

26th June | Online

Accelerating inference for every workload with TensorRT

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox