Guide To Catalyst – A PyTorch Framework For Accelerated Deep Learning


Catalyst is a PyTorch framework developed with the intent of advancing research and development in the domain of deep learning. It enables code reusability, reproducibility and rapid experimentation so that users can conveniently create deep learning models and pipelines without writing another training loop.

Catalyst framework is part of the PyTorch ecosystem – a collection of numerous tools and libraries for AI development. It is also a part of the Catalyst Ecosystem – an MLOps ecosystem that expedites training, analysis and deployment of deep learning experiments through Catalyst, Alchemy and Reaction frameworks respectively.

Highlighting features of Catalyst

  • It enables creating deep learning pipelines with just a few lines of code.
  • It is compatible with Python 3.6+ and PyTorch 1.3+ versions.
  • It enables the creation of a universal train/inference loop with features such as evaluation metrics, model checkpointing and early-stopping included.
  • All the source code, as well as environment variables, remain saved thereby enabling code reproducibility.
  • It enables creating configuration files for storing the model’s hyperparameters.
  • It also has a provision for ‘callbacks’ – ways to reuse parts of a train/inference pipeline with required customizations.
  • It supports some of the best deep learning R&D practices such as Stochastic Weight Averaging (SWA), Ranger optimizer, one-cycle training, fp16 precision, distributed training and so on.

Practical implementation

Here’s a demonstration of handling image classification task using Catalyst. We have used the well-known MNIST dataset having 10 output classes (for classifying images of handwritten digits from 0 to 9). The code has been implemented using Google colab with Python 3.7.10 and Catalyst 21.3 versions. Step-wise explanation of the code is as follows:

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.
  1. Install Catalyst library using pip command

!pip install -U catalyst

(-U option installs the updated version of library)

Download our Mobile App

  1. Import required libraries and modules
 import os
 from torch import nn, optim
 from import DataLoader
 from catalyst import dl, utils
 from import ToTensor
 from catalyst.contrib.datasets import MNIST 
  1. Define the neural network model, loss function and optimizer to be used

Here, we use a simple architecture comprising an input layer and a hidden layer each having 28 neurons while the output layer has 10 neurons since we have 10 possible output classes.

net = nn.Sequential(nn.Flatten(), nn.Linear(28 * 28, 10))

We have used the cross entropy loss function of PyTorch which combines LogSoftmax and NLLLoss in one class.

loss = nn.CrossEntropyLoss()

We are using Adam optimization algorithm 

opt = optim.Adam(net.parameters(), lr=0.02)  #’lr’ denotes learning rate

  1. Load data for training and validation

Using DataLoader() method of PyTorch, we load the MNIST dataset from contrib API of  Catalyst. We create a dictionary to load both training and validation sets at once by specifying the ‘train’ parameter of MNIST() as True or False respectively.

 data_loader = {
     "training": DataLoader(MNIST(os.getcwd(), train=True, download=True, 
     transform=ToTensor()), batch_size=32),
     "validation": DataLoader(MNIST(os.getcwd(), train=False,     
      transform=ToTensor()), batch_size=32),

ToTensor() method of ‘Data’ API converts a NumPy array to tensor representation.

  1. Define an experiment runner for handling experiments with supervised model using SupervisedRunner() method of Runners API.
exp_runner = dl.SupervisedRunner(input_key="features", output_key="logits", target_key="targets", loss_key="loss")
  1. Model training using Runners.train() method
     model=net,  #model to be trained
     criterion=loss, #loss function for training
     optimizer=opt, #optimizer for training

#dictionary with data loaders for training and validation

   #list with Catalyst callbacks
        #callback for accuracy computation
         dl.AccuracyCallback(input_key="logits", target_key="targets",  
         topk_args=(1, 3, 5)),
        #callback for plotting confusion matrix to the loggers
     logdir="./logs",  #path to output directory
  #name of loader to compute metrics and save the checkpoints
 #key to the metric name by which to select the checkpoints
 #flag indicating whether to minimize the valid metric specified above
     verbose=True,  #display status of model training to console
 #load the best checkpoint state as per the validation metrics

Sample output:

 Hparams (experiment): {}

 1/1 * Epoch (train): 100% 1875/1875 [00:20<00:00, 92.78it/s, 

accuracy=0.938, accuracy01=0.938, accuracy03=1.000, accuracy05=1.000, loss=0.098, lr=0.020, momentum=0.900]

 train (1/1) accuracy: 0.8802833557128906 | accuracy/std: 0.07081212792674185 | accuracy01: 0.8802833557128906 | accuracy01/std: 0.07081212792674185 | accuracy03: 0.9749666452407837 | accuracy03/std: 0.03580065525942475 | accuracy05: 0.9921166896820068 | accuracy05/std: 0.02176691186632642 | loss: 0.5139051675796509 | loss/std: 0.3664878010749817 | lr: 0.02 | momentum: 0.9
1/1 * Epoch (valid): 100%
313/313 [00:03<00:00, 83.65it/s, accuracy=0.875, accuracy01=0.875, accuracy03=1.000, accuracy05=1.000, loss=0.608, lr=0.020, momentum=0.900]
valid (1/1) accuracy: 0.8496999740600586 | accuracy/std: 0.08438007466043124 | accuracy01: 0.8496999740600586 | accuracy01/std: 0.08438007466043124 | accuracy03: 0.9573000073432922 | accuracy03/std: 0.04357493405029723 | accuracy05: 0.9886000156402588 | accuracy05/std: 0.019559607813700718 | loss: 0.8412032127380371 | loss/std: 0.5866137742996216 | lr: 0.02 | momentum: 0.9 

Note: By default, top-1, top-3 and top-5 relevant sets of samples are considered for metrics computation. So the output displays average accuracy as well as accuracy01, accuracy03 and accuracy05 accordingly.

These accuracy results for training and validation sets get stored in ‘training.csv’ and ‘validation.csv’ files which appear as follows:

Sample ‘training.csv’

Catalyst ouput1

Sample ‘validation.csv’

Catalyst output2


Refer to the following sources for understanding the Catalyst framework in detail:

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Nikita Shiledarbaxi
A zealous learner aspiring to advance in the domain of AI/ML. Eager to grasp emerging techniques to get insights from data and hence explore realistic Data Science applications as well.

Our Upcoming Events

24th Mar, 2023 | Webinar
Women-in-Tech: Are you ready for the Techade

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023 [AI100 Awards]

21 Jul, 2023 | New York
MachineCon USA 2023 [AI100 Awards]

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox