Recently, jiant 2.0 was released, which is an updated version of the natural language understanding toolkit jiant 1.x. jiant is a research library for natural language understanding that conducts multitask training and transfer learning experiments on English NLU tasks.

According to its developers, this toolkit enables modular and configuration-driven experimentation with state-of-the-art models as well as implements a broad set of tasks for probing, transfer learning, and multitask training experiments. 

The toolkit supports a wide range of tasks including the GLUE, SuperGLUE and XTREME benchmarks, and many of the newest transformers models such as BERT, RoBERTa, ALBERT, ELECTRA, XLM-R, among others.

What Is jiant 

jiant stands for Jiant Is An NLP Toolkit is an NLP research library written in Python. The library is built for researching general-purpose text understanding models and transfer learning. 

In simple words, jiant allows researchers to quickly experiment on a wide array of NLU tasks with the help of the natural language processing models as well as to conduct experiments on transfer learning and multitask training.

jiant supports several popular Transformer-based models implemented by Transformers package of Huggingface as well as non-Transformer models such as BiLSTMs. There are five core design principles of this library. They are mentioned below-

• Ease of use: This library allows the users to run a number of experiments using state-of-the-art models through a configuration-driven interface.
• Reproducibility: The library should provide features that support correct as well as reproducible experiments, including logging, restoring model state, among others.
• Availability of NLU tasks: The library should continue to grow a collection of tasks useful for natural language understanding research, especially popular evaluation tasks and tasks commonly used in pretraining and transfer learning.
• Availability of cutting-edge models: The toolkit should make implementations of state-of-the-art models available for experimentation
• Open source: jiant is free to both use and contribute.

What’s New

Built with PyTorch, jiant 2.0 was rebuilt from the ground up to support cutting-edge NLP research. In order to support the changing needs of NLP researchers, the jiant toolkit has undergone some of the major changes and is now built with transformers models and datasets. 

jiant 2.0 has a new training pipeline to facilitate modern experimental workflows. According to the developers, this toolkit is no longer a framework and can be called as a modular library.  

Why Use jiant

The first point to consider why using this toolkit is that jiant is written in Python and supports more than 50 natural language understanding tasks, multitask learning and transfer learning. The library both integrates with datasets to manage task data and with transformers to manage models and tokenizers.

While doing computational research, a unified API is needed for running experiments with different models, tasks, or run configurations rather than making ad-hoc tweaks for different experiments or running experiments with different scripts that may result in duplicated codes.

For such reasons, jiant is designed so that it can facilitate large-scale, replicable, configuration-driven experiments from a single standardised interface. This toolkit brings several important components into the table. Some of them are mentioned below-

Tasks

jiant supports more than 50 natural language understanding tasks built-in part on Hugging Face’s datasets. These include the GLUE, SuperGLUE, and XTREME benchmark tasks, edge probing tasks, and many more. 

Models

jiant supports BERT, RoBERTa, XLM-R, ALBERT, BART, ELECTRA, etc. based on Hugging Face’s transformers and tokenizers libraries. The toolkit comprises of a text encoder (e.g. BERT) and supports a wide variety of task heads including classification, regression, multiple-choice, span comparison, span prediction, multi-label span comparison, tagging, question answering and masked language modelling. 

Runner

jiant provides a Runner implementation, which handles the training loop for an experiment (similar to transformers Trainer). jiant’s Runner natively supports multitask training, and this includes considerations like different task-sampling methods and different batch sizes or gradient accumulation steps per task. 

How To Use It

jiant uses proportional sampling from each task as default. For the Python version, jiant can be used writing the following code-

from jiant.proj.simple import runscript as run

import jiant.scripts.download_data.runscript as downloader