Google launched the upgraded version of TensorFlow (TF) today. The new upgrade, version 2.7.0, comes with several new and improved features like improved debugging experience, public convolution, data service auto-sharding, and more.
This article will showcase some of the major updates and features of TensorFlow 2.7.0.
Improved TensorFlow Debugging Experience
In the previous version, TensorFlow error stack traces involved many internal frames, which could be challenging to read through and not be actionable for end users. In the latest version, TensorFlow filters internal frames in most errors, thereby keeping stack traces short, readable, and focused on actionable information for end-users.
This behavior can be disabled by entering tf.debugging.disable_traceback_filtering(), and can be re-enabled via tf.debugging.enable_traceback_filtering(). The Google team said if you are debugging a TensorFlow-internal issue, make sure to disable traceback filtering. To enable traceback filtering: use tf.debugging.is_traceback_filtering_enabled().
Also, the team has improved the informativeness of error messages raised by Keras by adding the full list of ‘argument values’ passed to the layer in every exception.
The new feature is available with Python 3.7 and above versions.
New Style Variables
In the latest version, the Google team has introduced the tf.compat.v1.keras.utils.track_tf1_style_variables decorator. It enables using large classes of ‘tf1-style variable_scope, get_variable,’ and ‘compat.v1.layer’ based components from within TF2 models running with TF2 behavior enabled.
TensorFlow Data
TF data service now supports auto-sharding. For instance, users can specify the sharding policy with tf.data.experimental.service.ShardingPolicy enum. Enum, short for ‘enumerated,’ is a data type that consists of predefined values.
It can be one of OFF (parallel_epochs), DYNAMIC ( distributed_epoch), or one of the auto-sharding policies: DATA, FILE, FILE_OR_DATA, or HINT (corresponding to values of ‘tf.data.experimental.AutoShardPolicy’).
Auto-sharding (static sharding) requires the number of tf.data service workers to be fixed. Developers and users need to specify the worker addresses in TensorFlow.data.experimental.DispatcherConfig.
The TensorFlow dataset (tf.data.experimental.service.register_dataset) now accepts optional compression arguments.
Keras
The Keras layers now include a public convolution_op method. It is used to simplify the implementation of Convo subclasses. There are two ways to use this new technique.
The first method is to use it directly in your call method:
Second option: you can override convolution_op:
In the latest version, the team has added the merge_state() method to tf.keras.metrics.Metric for use in distributed computations, alongside ‘sparse’ and ‘ragged’ options to ‘tf.keras.layers.TextVectorization’ to allow for ‘SparseTensor’ and ‘RaggedTensor’ outputs from the layer.
RPC APIs
In TF 2.7.0 version, Google has introduced APIs to create a gRPC based server to register ‘tf.function’ methods and a gRPC client to invoke remote registered methods. gRPC is the modern, lightweight communication protocol from Google. RPC APIs are typically intended for multi-client setups like servers, and clients are started in separate binaries independently.
Bug Fixes and other changes
The TensorFlow version 2.7.0 comes with many bug fixes:
TF Core
- Random number generation (RNG) system now comes with new functions to explicitly select the RNG algorithm, a stateless version of dropout, and the generator can be created inside the scope of Parameter Server Strategy.
- Now you can add an experimental session (tf.experimental.disable_functional_ops_lowering) which disables functional control flow op lowering optimization. This function is useful when executing within a portable runtime where control flow op kernels may not be loaded because of selective registration.
- Also, you can add a new experimental argument to the static hash table to create the table in anonymous mode. Here, the table resource can only be accessed via resource handles (not resource names) and will be removed automatically when all resource handles pointing to it are gone.
TF Dataset
- In the latest version, you can now introduce the TF data API which provides random access for input pipelines that consists of transformations that support random access. This includes: tf.data.Dataset.from_tensor_slices,tf.data.Dataset.shuffle, tf.data.Dataset.batch, tf.data.Dataset.shard, tf.data.Dataset.map, and tf.data.Dataset.range.
- You can promote tf.data.Options.experimental_deterministic API to tf.data.Options.deterministic and deprecate the experimental endpoint.
- You can move autotuning options from ‘tf.data.Options.experimental_optimization.autotune’ to a newly created ‘tf.data.Options.autotune.’ and remove support for tf.data.Options.experimental_optimization.autotune_buffers.
- You can add support for user-defined names of TF data core Python API, which can be used to disambiguate ‘TF data events’ in TF Profiler Trace Viewer.
- Also, you can promote ‘tf.data.experimental.sample_from_datasets’ API to ‘tf.data.Dataset.sample_from_datasets’ and deprecate the experimental endpoint.
TF SavedModel
In the latest version of TensorFlow, custom gradients are now saved by default. The saved_model_cli’s – inputs are now restricted to
python literals to avoid code injection.
XLA
In TensorFlow 2.7.0, you can add a new API that allows custom call functions to signal errors. The team said that the old API would be deprecated in a future release. Check out more details here.
XLA: GPU reductions are deterministic by default. It works with Horovod (OSS contribution by Trent Lo from NVIDIA)
Lastly, the teams said when saving a model, not specifying a namespace whitelist for ‘custom ops’ with a namespace will now ‘default to allowing’ rather than ‘rejecting’ them all.
Wrapping up
This is exciting news for the open-source developer ecosystem. The TensorFlow team at Google Brain Research, alongside other contributors, have been working to enhance and update its popular machine learning platform.
However, the recent release of version 2.7.0 comes at a time when the popularity of TensorFlow is declining after Facebook PyTorch burst into the scene. Hopefully, the recent update will give developers a seamless experience and regain its popularity once again.
