PyTorch Releases v1.9: All New Features & Updates

Open-source machine learning framework PyTorch has released its version 1.9 just two months after the release of version 1.8. The new release consists of 3,400 new commits made by 398 contributors. 

Highlights:

  • Improvements to functions such as torch.linalg, torch.special, and Complex Autograd to support scientific computing
  • Beta release of Mobile Interpreter to execute PyTorch programs in edge devices
  • Upstreaming of TorchElastic into Pytorch Core
  • Major updates to the PyTorch RPC framework
  • New APIs for performance optimization
  • Support for distributed training, GPU utilization, and SM efficiency in the PyTorch Profiler.
https://twitter.com/PyTorch/status/1404838218277740549?s=20

Increased support for scientific computing

In the new version, torch.linalg has been moved to a stable release. This module extends PyTorch support for linear algebra by implementing several functions such as NumPy’s linear algebra module, torch.linalg.matrix_norm and torch.linalg.householder_product. 

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

The Complex Autograd feature has also been promoted to a stable state after being released as a beta in v1.8. In the latest release, the team has extended support for Complex Autograd for 98 percent operators, added more OpInfos, and TorchAudio has been migrated to native complex tensors.

Complex Autograd provides users the ability to calculate complex gradients and work with complex variables. It is an important feature for users who need to work with complex numbers in PyTorch toolkits such as TorchAudio, ESPNet, FastMRI, and Asteroid. 

PyTorch 1.9 also includes torch.use_determinstic_algorithms option for debugging and writing reproducible programs.

The torch.special module, which contains functions for scientific computing and helps in working with distributions like iv, ive, erfcx, logerfc, and logerfcx, has been made available in beta. Along with torch.special module, nn.Model parameterization is also now available in beta. It helps to reduce the space required for parameters without using any special optimization methods.

PyTorch Mobile

Mobile Interpreter in beta to execute PyTorch programs in edge devices is one of the major announcements. Mobile Interpreter will significantly reduce the binary size as compared to the current on-device runtime. “The current pt size with MobileNetV2 in arm64-v8a Android is 8.6 MB compressed and 17.8 MB uncompressed. Using Mobile Interpreter, we are targeting at the compressed size below 4 MB and uncompressed size below 8MB,” the team said in the blog.

PyTorch is releasing a new PyTorch Video library-based video app and an update on the speech recognition app based on torchaudio and wave2vec model. With the addition of these two apps, PyTorch now offers a full demo apps suite that cover image, text, audio, and video.

Distributed training

TorchElastic is a coordinator for PyTorch worker processes that handles scaling events. It was open-sourced more than a year ago and has been used in various distributed torch use-cases such as deepspeech.pytorch, pytorch-lighting, and Kubernetes CRD. In v1.9, TorchElastic has been made part of PyTorch core.

Other beta features made available in the distributed package include–CUDA support in RPC, ZeroRedundancyOptimizer, and support for profiling distributed collectives.

Performance optimization

Module Freezing is a process for optimising and specialising programs; freezing is a recommended practice for model deployment. It inlines module parameters and attribute values as constants into the TorchScript internal representation.

PyTorch Profiler, Inference mode API, and torch.package have been graduated to beta.

Other updates

Along with the release of PyTorch v1.9, the team also announced updates to a couple of PyTorch libraries which include new releases for domain libraries such as TorchVision, TorchText, and TorchAudio. 

Highlights:

  • TorchVision: Addition of new SSD and SSDLite models, quantized kernels for object detection, and iOS support
  • TorchAudio: Several new additions including wav2vec 2.0 model, options for quality control in sampling, and performance improvements in spectral operations and resampling.
  • TorchText – Introduction of a new high-performance Vocab module for common functional APIs for NLP workflows. 

More Great AIM Stories

Shraddha Goled
I am a technology journalist with AIM. I write stories focused on the AI landscape in India and around the world with a special interest in analysing its long term impact on individuals and societies. Reach out to me at shraddha.goled@analyticsindiamag.com.

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

Conference, in-person (Bangalore)
Cypher 2023
20-22nd Sep, 2023

3 Ways to Join our Community

Whatsapp group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our newsletter

Get the latest updates from AIM