Open-source machine learning library inspired by Torch — Pytorch has been emerging as one of the critical deep learning tools. The library has several intuitive features such as manipulating computational graphs on the go; easier learn methods and other such. In one of our previous articles, we had discussed the reasons why Pytorch is becoming one of the go-to tools for deep learning developers.
In October, the team at FAIR introduced the PyTorch-based codebase, PySlowFast at ICCV 2019. This time, the researchers at Facebook AI Research (FAIR) open sourced the codebase, PySlowFast, which is an open-source video understanding codebase which provides state-of-the-art video classification models.
About PySlowFast
According to the researchers, the SlowFast networks can be described as a single stream architecture that operates at two different framerates. The main motive behind the PySlowFast codebase is to provide a high-performance, light-weight PyTorch codebase, which can give the video backbones for video understanding research on different tasks such as classification and detection. PySlowFast includes implementations of some of the backbone network architectures which are SlowFast, SlowOnly, C2D, I3D, Non-Local Neural Network.
How It Works
The SlowFast model involves dual pathways which are fused by lateral connections. By treating the raw video at different temporal rates of the pathways, the two pathways gain the ability to have their expertise in video modelling. The pathways are mentioned below:
- Slow pathway: The Slow pathway is a convolutional model that works on a clip of a video as a spatiotemporal volume. Slow pathway operates at a low frame rate, to capture spatial semantics. It has been designed to capture semantic information that can be given by images or a few sparse frames and also operates at low frame rates with slow refreshing speed.
- Fast pathway: The Fast pathway is another convolutional model with properties like high frame rate, high temporal resolution features, and low channel capacity. Fast pathway operates at a high frame rate, to capture motion at fine temporal resolution. It is responsible for obtaining rapidly changing motion by running at fast refreshing speed and high temporal resolution. Also, the Fast pathway can be made very lightweight by reducing its channel capacity, yet can learn useful temporal information for video recognition.
Installation
- Clone the PySlowFast Video Understanding repository:
git clone https://github.com/facebookresearch/slowfast
- Add this repository to $PYTHONPATH:
export PYTHONPATH=/path/to/SlowFast/slowfast:$PYTHONPATH
- After having the above dependencies, run:
git clone https://github.com/facebookresearch/slowfast
cd SlowFast
python setup.py build develop
Wrapping Up
PySlowFast is mainly designed to support the rapid implementation and evaluation of novel video research ideas. The researchers at Facebook AI have been striving hard to create new benchmarks in video understanding, multi-object mesh prediction, and object recognition. The requirements while installing the codebase include PyTorch 1.3, NumPy, Python >= 3.6, GCC >= 4.9, etc.
Before PySlowFast codebase, the researchers had unveiled Detectron, which is also a high-performance codebase for object detection, covering both bounding box and object instance segmentation outputs. The researchers themselves use the codebase in-house on several state-of-the-art research projects such as Feature Pyramid Networks for object detection, mask R-CNN, detecting and recognizing human-object interactions, and a focal loss for dense object detection.
