PyTorch Profiler: Major Features & Updates

The Profiler collects both GPU and framework related information.

Recently, Python’s popular open-source machine learning library, PyTorch announced its new performance debug profiler, PyTorch Profiler, along with its 1.8.1 version release. The new tool — developed as a part of a collaboration between tech giants Facebook and Microsoft — enables accurate and efficient performance analysis in large scale deep learning models.

Behind PyTorch Profiler

With a new module namespace torch.profiler, PyTorch Profiler is the successor of PyTorch autograd profiler. This new tool uses a new GPU profiling engine — built using the NVIDIA CUPTI APIs — and can capture the GPU kernel events with high fidelity. PyTorch includes a simple profiler API that is useful when determining the most expensive operators in the model.

The Profiler collects both GPU and framework related information. The tool then correlates them, performs automatic detection of bottlenecks in the model, generates recommendations on how to resolve these bottlenecks, and visualise them. The new Profiler API is natively supported in PyTorch. Users can profile their models using this API, without installing any additional packages and get results immediately in TensorBoard with the new PyTorch Profiler plugin.  

PyTorch profiler accepts several parameters. some of the most valuable parameters to analyse the execution time are:

  • record_shapes – whether to record shapes of the operator inputs;
  • profile_memory – whether to report the amount of memory consumed by the model’s Tensors
  • use_cuda – whether to measure execution time of CUDA kernels.

To analyse the memory consumption, the PyTorch Profiler can show the amount of memory used by the model’s tensors allocated during the execution of the model’s operators. 

Importance of Profiler In ML

Analysing and performance improvement of deep learning models are some of the existing challenges researchers and developers face as the model’s size increases. According to sources, for a long time now, PyTorch users have a hard time solving these challenges due to the lack of available tools. The standard performance debugging tools that provide GPU hardware level information missed the PyTorch-specific context of operations. 

To recover the missed information, users needed to combine multiple tools or manually add minimum correlation information to make sense of the data. An autograd profiler, known as torch.autograd.profiler, could capture information about PyTorch operations but does not capture detailed GPU hardware-level information or support visualisation. 

The PyTorch Profiler (torch.profiler) is a tool that can capture both information about PyTorch operations and capture detailed GPU hardware-level information. 

Advantages of Profiler

According to a few Reddit users, the new tool will be more useful than its previous version and NVVP (Nvidia visual profiler) as it provides profile data-loading, processing, matching of GPU operations to your PyTorch modules and more.  

The advantages of PyTorch Profiler are as follows:

  • To gain insight regarding the operations run inside a model.
  • To diagnose performance issues and optimise deep learning models.
  • To analyse the performance profile.
  • Identifies the bottlenecks and eliminates them by following the recommendations.
  • Lets users inspect the cost of different operators inside the model – both on the CPU and GPU.
  • To measure the time and memory consumption of the model’s operators. 

Download our Mobile App

Ambika Choudhury
A Technical Journalist who loves writing about Machine Learning and Artificial Intelligence. A lover of music, writing and learning something out of the box.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

Career Building in ML & AI

31st May | Online

Rakuten Product Conference 2023

31st May - 1st Jun '23 | Online

MachineCon 2023 India

Jun 23, 2023 | Bangalore

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox