Active Hackathon

Face Emotion Recognizer In 6 Lines of Code

In this article, I’ll be discussing how to create a face emotion recognizer using ‘FER’ library from python.

Whatever we feel at heart is understood by our facial expressions. Facial expressions are a vital mode of communication. It is said that any person’s behaviour is controlled by his/her face. Social Media to video chat applications our emotions are tracked everywhere. Medical research has also used them widely. Mainly to understand mental health by analyzing emotions. AI has made this possible. Computer Vision has reached new heights from recognising only faces to decoding face gestures. Face recognition has been in this field for ages but have you ever wondered how interesting it would be to decode facial expressions mainly happy, sad, fear, anger, surprise, neutral, disgust. 

In this article, I’ll be discussing how to create a face emotion recognizer using ‘FER’ library from python.


Sign up for your weekly dose of what's up in emerging technology.

FER library

Facial Expression Recognition with a deep neural network using Tensorflow and Keras libraries implemented in python. Dataset used is from Kaggle competition Challenges in Representation Learning: Facial Expression Recognition Challenge. 


OpenCV 3.2+

Tensorflow 1.7+

Python 3.6+


pip install fer

Predicting emotions over the static image

from fer import FER
import matplotlib.pyplot as plt 
img = plt.imread("happy.jpg")
detector = FER(mtcnn=True)

[OrderedDict([(‘box’, (172, 31, 93, 93)), (’emotions’, {‘angry’: 0.0, ‘disgust’: 0.0, ‘fear’: 0.0, ‘happy’: 1.0, ‘sad’: 0.0, ‘surprise’: 0.0, ‘neutral’: 0.0})])]

<matplotlib.image.AxesImage at 0x1ad9f99fd30>

Detector returns a list containing the Ordered dictionary of bounding box notations where the face is detected and all the 7 emotions in decimals values from 0 to 1. The FER contains the Keras model built with convolutional neural networks and weights saved in HDF5 model. Alternatively, there’s Peltarion API which could be used in the backend in place of Keras model. MTCNN stands for multi cascade convolutional network. It is an advanced technique for detecting faces. If mtcnn=False then by default OpenCV Haar Cascade Classifier is used. Lastly, the detect_emotions() function is called to classify the emotion into ‘happy’, ’sad’, ‘disgust’, ‘anger’, ‘fear’, ‘neutral’ with values for each.

If we wish to only want the emotion with the highest score we can directly do that with top_emotion() function.

emotion, score = detector.top_emotion(img)

happy 1.0

Lets look at other emotions:

img = plt.imread("sad.jpg")
detector = FER()

(‘sad’, 0.91)

Similarly, the rest of the emotions are shown.

(‘angry’, 0.96)

(‘neutral’, 0.88)

(‘surprise’, 0.95)

(‘fear’, 0.92)

The ‘fer’ library has a separate module for analysis of facial emotions in videos. This extracts frames and performs emotion analysis using video.analyze() function over detected faces.

from fer import Video
from fer import FER
video_filename = "D:/python/YouTube.mp4"
video = Video(video_filename)
# Analyze video, displaying the output
detector = FER(mtcnn=True)
video.analyze(detector, display=True)

Storing the emotion and detector face

For detected face bounding boxes is drawn and the emotion having the highest value is shown with brighter text colour than others. 

import cv2
from fer import FER
detector = FER(mtcnn=True) 
image = cv2.imread("andrew_ng.jpg")
result = detector.detect_emotions(image)

As discussed above the result list will return bounding box and emotion values which are stored in separate arrays.

bounding_box = result[0]["box"]
emotions = result[0]["emotions"]
Bounding around face is drawn
cv2.rectangle(image,(bounding_box[0], bounding_box[1]),
(bounding_box[0] + bounding_box[2], bounding_box[1] + bounding_box[3]),(0, 155, 255), 2,)

Now the emotion along with scores are highlighted

for idx, (emotion, score) in enumerate(emotions.items()):
    color = (211, 211, 211) if score < 0.01 else (255, 0, 0)
    emotion_score = "{}: {}".format(
          emotion, "{:.2f}".format(score) if score > 0.01 else ""
            (bounding_box[0], bounding_box[1] + bounding_box[3] + 30 + idx * 15),cv2.FONT_HERSHEY_SIMPLEX,0.5,color,1,cv2.LINE_AA,)
cv2.imwrite("emotion.jpg", image)


Face emotion detectors with a few lines of code using the fer library is easy to implement and can be integrated with other applications. Robotics has evolved over the years and many bots can now mimic human reactions. Another use case could be online proctored exams where people’s reactions can be tracked and monitored.

The complete code of the above implementation is available at the AIM’s GitHub repository. Please visit this link to find the notebook of this code.

More Great AIM Stories

Jayita Bhattacharyya
Machine learning and data science enthusiast. Eager to learn new technology advances. A self-taught techie who loves to do cool stuff using technology for fun and worthwhile.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM