Beginner’s Guide To TensorFlow Eager Execution

The modules in TensorFlow are developed to assist its core principle — make tensors flow through a graph — just as the name suggests. From the numbers that are used to represent the models to the operations themselves, everything is a part of a graph. And this creates some hurdles for those who are starting out with machine learning.

This graph-based functioning creates problems when one has to debug their code written with TensorFlow. Writing a perfect machine learning model that behaves well is a hyperbole. And, any developer would like to sneak in on to the code in between and monitor it with the help of partial results.

That is why the team at TensorFlow came up with an Eager mode. 

What Is Eager Mode?

In this mode, a practitioner has to run a single line of code to enable the eager execution module on TensorFlow and keep a track of their code. This makes it easy to get started with TensorFlow and debug models. In Tensorflow 2.0, eager execution is enabled by default.

TensorFlow eagerly executes the operations thus allowing the developer to see the results on the go rather than wait for the whole model to be executed. 

Download our Mobile App

<code>tf.executing_eagerly()</code>

Enabling eager execution changes how TensorFlow operations behave — now they immediately evaluate and return their values to Python. 

For example, performing a simple matrix multiplication would require the following:

import tensorflow as tf

x = [ [1,2],

[3,4] ]

m = tf.matmul(x,x)

print(m)

This would print an output which is not so insightful:

Tensor(“MatMul:0”, shape = (2,2), dtype = int32); which is not so insightful.

To make any sense of what a certain chunk of the model is undergoing, one would need to run a session:

with tf.Session() as sess:

print(sess.run(m))

[ [7 10]

[15 22] ]

So, to pull out the values one has to wrap the operation in a session. This becomes tedious with complicated tasks.

However, when eager mode is enabled:

tf.enable_eager_execution()

tf.executing_eagerly()

The above lines followed by

print(m)

Would give the following output:

tf.Tensor( [ [ 7 10] [ 15 22] ], shape= (2,2), dtype = int32)

Why Eager Mode Matters

The fundamental operations in a typical neural network can be reduced to a bunch of addition and multiplication operations. Whether it is a convolutional neural network to recognise images or a language model to perform sentiment analysis, these basic arithmetic operations play a huge role. As shown above, if a 2×2 matrix needs a session wrapper to pull out numbers, imagine a large model running on high dimensional data. The whole process becomes exhaustive and inefficient. 

Eager execution works nicely with NumPy. NumPy operations accept tf.Tensor arguments. TensorFlow math operations convert Python objects and NumPy arrays to tf.Tensor objects. The tf.Tensor.numpy method returns the object’s value as a NumPy ndarray.

Let’s take a look at two examples where eager execution helps the developer:

Tensor Operations

When Eager Execution is disabled, the tensor manipulation is limited. For example, tensor operations will yield a tensor object and not the result of the operations. Refer to the code example below.

Model Output Before Training

Even without training, call the model and inspect the output in eager execution. A TensorFlow model needs to be compiled and trained before it can produce an output but with Eager Execution the training can be by-passed and the output can be inspected for a given input. 

Along with the advantages discussed above, the eager mode can also help the developers to call the model without training and inspect the output in eager execution.

The benefits of Eager execution, as told by the developers at TensorFlow, can be summarised as follows: 

• Quickly iterate on small models and small data.
• Easier debugging.
• Support for dynamic models using easy-to-use Python control flow
• Strong support for custom and higher-order gradients
• Almost all of the available TensorFlow operations.

Debugging lies at the core of any programming task. Write, check for errors, debug and repeat. The sooner the errors are identified, the sooner the chances of a model getting deployed. The eager mode in TensorFlow is one such option to make use of this widely opted framework in a more efficient way.