Deep Learning is generally considered a black box technique, given that one cannot analyse how it is working in the back-end. The creator can create a deep neural network, compile it, and then fit it on their data – if done correctly, it will work using neurons transferring the information using different layers and all the activations and other important hyperparameters. But one can’t really visualise how data is being transferred or how the model is learning. We have broken down the visualisations – how neural networks really look; or the closest they come to it, with examples from several researchers and data scientists.
Deep Learning Visualisation Models created in Python:
Implementing Unity to create Neural Networks
Unity is an AI platform for creating and operating real-time 3D content. It is an excellent tool for developers and designers to make imaginations come to life. Unity’s comprehensive software solutions can create, run, and monetise interactive, real-time 2D and 3D content.
View a demonstration video here.
Interactive 3D Neural Network Visualizer
The visualiser is Unity’s tool to make the structure of neural networks more visually clear with increased interactive ways to edit architectural parameters. It is an efficient tool to study how the layers of a deep neural network or CNN are connected and visualise the number of weights/calculations involved in the model.
There are two “modes”:
- Constructing the CNN in the Unity Editor by combining the prefabs and adjusting parameters.
- Loading a JSON file with weights and activations from an actual TensorFlow model to be displayed, which has to be set up and converted first.
Stefan Sietzen’s Visualisation
Source: Stefan Sietzen’s
Stefan Sietzen is a deep learning engineer at Kaleido AI and a freelance video creator with an interest in video computing. Sietzen’s project is a 3D visualisation of a CNN showing all the layers. He has built a Python script that trains a CNN in TensorFlow; it exports a JSON file that the CNN visualised can read automatically.
Step 1: Loading TensorFlow checkpoints to build the graph
The researcher is loading and visualising Tensorflow weights and feature map activations from a custom JSON format. This has to be converted with a python script from original .ckpt binary files.
View a demonstration video here.
Step 2: Building the CNN in the Unity Editor
The next step is designing a CNN in the Unity Editor by combining pre-configured Game Objects (Prefabs) to build the graph. A large number of parameters in this mode allows the user to collapse and expand each layer differently. In addition, this makes it possible to seamlessly transition between showing filters as edges or showing all the calculations.
Edge bundled layers with one convolutional layer displaying the calculations.
Zoomed in calculation modes.
Fully connected layer displaying all weights.
Fully connected layer where only one edge leaves each input feature map.
Default view : Displaying the real number of filter weights as edges.
Smaller feature maps in full number.
Houdini
View a demonstration video of training a recurrent neural network (RNN) to stabilize an inverted double pendulum. This is created using Houdini, a 3D animation software application developed by SideFX.
Keras Model Learning
Keras is an easy-to-use Python library for developing and evaluating deep learning models. It is built on top of TensorFlow 2 and can scale to large clusters of GPUs, or an entire TPU pod, making it easy to create neural networks using the software.
In the following demonstration, Himanshu Sharma, a data science master, has used Keras’ deep replay for the neural network training process.
Source: Himanshu Sharma111
Producing a Latex Code in Plot NeuralNet
Unlike softwares that use architecture code, PlotNeuralNet works by separating and defining the network code for the program to consume. The program uses this to output the final diagram.
CNN architecture: PlotNeuralNet (Github page)
Holistically-Nested Edge Detection (view on Overleaf)
FCN-32 (view on Overleaf)
Source: Haris Iqbal on GitHub
Illustrations like these help in visualising how a neural network really looks and works, and the different elements that go with it. But, as we noted earlier, there is still plenty left to learn about deep learning.
