|
Listen to this story
|
Intel engineers in the PyTorch open-source community have created a new Intel® Extension for PyTorch*, which they claimed maximises deep learning inference and training performance on Intel CPUs. Fan Zhao, engineering manager at Intel, shared in a post that Intel® Extension for PyTorch*optimises for both imperative mode and graph mode. The optimisations are performed for three key pillars of PyTorch – operators, graph and runtime.
She said in the same post that during execution, Intel® Extension for PyTorch* will override a subset of ATen operators with their optimised counterparts in the extension. It will offer an additional set of custom operators and optimisers for popular use cases. In graph mode, additional graph optimisation passes will be applied that will maximise the performance of the underlying kernel implementations in the extension.
“Runtime optimisations are encapsulated in the runtime extension module, which provides a couple of PyTorch frontend APIs for users to get finer-grained control of the thread runtime,” she added.
Image: Accelerating PyTorch with Intel® Extension for PyTorch* | by Fan Zhao | PyTorch | May, 2022 | Medium
Performance
The blog gave some examples of performance gains with Intel® Extension for PyTorch*. The numbers were measured on Intel(R) Xeon(R) Platinum 8380 CPU @ 2.3 GHz.
Image: Accelerating PyTorch with Intel® Extension for PyTorch* | by Fan Zhao | PyTorch | May, 2022 | Medium
In the future, the team behind this wants to bring users additional performance boost on Intel CPUs in an agile and fast fashion. Zhai added that the team would upstream most of the optimisations to stock PyTorch over time. They will also land new optimisations and features for the latest Intel hardware for users to get their hands on them faster.
To know more, click here.
