Now Reading
How Facebook’s New Open Sourced Codebase PySlowFast Is Helping Them Understand Videos Better

How Facebook’s New Open Sourced Codebase PySlowFast Is Helping Them Understand Videos Better


Open-source machine learning library inspired by Torch — Pytorch has been emerging as one of the critical deep learning tools. The library has several intuitive features such as manipulating computational graphs on the go; easier learn methods and other such. In one of our previous articles, we had discussed the reasons why Pytorch is becoming one of the go-to tools for deep learning developers. 

In October, the team at FAIR introduced the PyTorch-based codebase, PySlowFast at ICCV 2019. This time, the researchers at Facebook AI Research (FAIR) open sourced the codebase, PySlowFast, which is an open-source video understanding codebase which provides state-of-the-art video classification models.

About PySlowFast

According to the researchers, the SlowFast networks can be described as a single stream architecture that operates at two different framerates. The main motive behind the PySlowFast codebase is to provide a high-performance, light-weight PyTorch codebase, which can give the video backbones for video understanding research on different tasks such as classification and detection. PySlowFast includes implementations of some of the backbone network architectures which are SlowFast, SlowOnly, C2D, I3D, Non-Local Neural Network



How It Works

The SlowFast model involves dual pathways which are fused by lateral connections. By treating the raw video at different temporal rates of the pathways, the two pathways gain the ability to have their expertise in video modelling. The pathways are mentioned below:

  • Slow pathway: The Slow pathway is a convolutional model that works on a clip of a video as a spatiotemporal volume. Slow pathway operates at a low frame rate, to capture spatial semantics. It has been designed to capture semantic information that can be given by images or a few sparse frames and also operates at low frame rates with slow refreshing speed.
  • Fast pathway: The Fast pathway is another convolutional model with properties like high frame rate, high temporal resolution features, and low channel capacity. Fast pathway operates at a high frame rate, to capture motion at fine temporal resolution. It is responsible for obtaining rapidly changing motion by running at fast refreshing speed and high temporal resolution. Also, the Fast pathway can be made very lightweight by reducing its channel capacity, yet can learn useful temporal information for video recognition. 

Installation

  • Clone the PySlowFast Video Understanding repository:

git clone https://github.com/facebookresearch/slowfast

  • Add this repository to $PYTHONPATH:

export PYTHONPATH=/path/to/SlowFast/slowfast:$PYTHONPATH

  • After having the above dependencies, run:

git clone https://github.com/facebookresearch/slowfast

cd SlowFast

See Also
Artificial Intelligence Risks Losing Trust If It Doesn't Offer Explanations

python setup.py build develop

Wrapping Up

PySlowFast is mainly designed to support the rapid implementation and evaluation of novel video research ideas. The researchers at Facebook AI have been striving hard to create new benchmarks in video understanding, multi-object mesh prediction, and object recognition. The requirements while installing the codebase include PyTorch 1.3, NumPy, Python >= 3.6, GCC >= 4.9, etc. 

Before PySlowFast codebase, the researchers had unveiled Detectron, which is also a high-performance codebase for object detection, covering both bounding box and object instance segmentation outputs. The researchers themselves use the codebase in-house on several state-of-the-art research projects such as Feature Pyramid Networks for object detection, mask R-CNN, detecting and recognizing human-object interactions, and a focal loss for dense object detection.


Enjoyed this story? Join our Telegram group. And be part of an engaging community.


Provide your comments below

comments

What's Your Reaction?
Excited
1
Happy
0
In Love
0
Not Sure
0
Silly
0
Scroll To Top