TensorFlow v2.9 released: Major highlights

Just three months after the release of version 2.8, TensorFlow has now announced the latest 2.9 version. The main highlights of this release are performance enhancement with oneDNN and the release of a new API for model distribution, called DTensor, for seamless movement between data and model parallelism.

TensorFlow integrated the oneDNN performance library to achieve superior performance on Intel CPUs. TensorFlow has been offering experimental support for oneDNN since the release of version 2.5, offering four times performance improvement. In its latest release, oneDNN optimisations will be turned on by default on Linux x86 packages and for CPUs with neural-network-focused hardware features, such as AVX512_VNNI, AVX512_BF16, AMX, and others, which are found on Intel Cascade Lake and newer CPUs.

Another major feature of this release is the introduction of DTensorflow, which is a new API for models to move from data parallelism to single program multiple data (SPMD) based model parallelism, including spatial partitioning. A developer would now have tools to train models where the model inputs are large enough not to fit on a single device. It is a device-agnostic API, allowing the same model code to be used on CPU, GPU or TPU – also including models partitioned across device types. It also removes the coordinator and leaves the task to drive its locally attached devices. This allows model scaling without impacting the startup time.

Earlier, the model code was written around replicas, but with DTensor, the model code is written from a global perspective and per replica code is delivered by DTensor runtime.

Other major announcements include the Window Subsystem for Linux for developers to run a Linux environment directly on Windows without needing a virtual machine or dual boot setup.

With version 2.8, TensorFlow introduced an API to make ops deterministic. Determinism refers to running an op multiple times with the same inputs to obtain the exact same output each time. In version 2.9, the determinism performance is improved.

TensorFlow has also released a new experimental version of Keras Optimizer API for offering a unified and expanded catalogue of built-in and customised optimisers.