Article complete

What A Neural Network Really Looks Like

A deep neural network is created by sandwiching “hidden” layers between the input and the output.

What A Neural Network Really Looks Like

Key takeaways

  • Deep learning is often viewed as a black box, making it difficult to understand its inner workings.
  • Neural networks consist of hidden layers that process information between input and output.
  • Unity provides tools for developers to create and visualise neural networks in interactive 3D.
  • The Interactive 3D Neural Network Visualizer enhances understanding of neural network architecture and calculations.

Takeaways drafted with AI from the article text.

Deep Learning is generally considered a black box technique, given that one cannot analyse how it is working in the back-end. The creator can create a deep neural network, compile it, and then fit it on their data - if done correctly, it will work using neurons transferring the information using different layers and all the activations and other important hyperparameters. But one can't really visualise how data is being transferred or how the model is learning. We have broken down the visualisations - how neural networks really look; or the closest they come to it, with examples from several researchers and data scientists. 

Deep Learning Visualisation Models created in Python:

Implementing Unity to create Neural Networks

Unity is an AI platform for creating and operating real-time 3D content. It is an excellent tool for developers and designers to make imaginations come to life. Unity's comprehensive software solutions can create, run, and monetise interactive, real-time 2D and 3D content.

View a demonstration video here. 

Interactive 3D Neural Network Visualizer

The visualiser is Unity's tool to make the structure of neural networks more visually clear with increased interactive ways to edit architectural parameters. It is an efficient tool to study how the layers of a deep neural network or CNN are connected and visualise the number of weights/calculations involved in the model.

There are two "modes":

  1. Constructing the CNN in the Unity Editor by combining the prefabs and adjusting parameters.
  2. Loading a JSON file with weights and activations from an actual TensorFlow model to be displayed, which has to be set up and converted first.
https://twitter.com/ReJ/status/1358665255522598913?s=20

Stefan Sietzen’s Visualisation

Source:  Stefan Sietzen’s

Stefan Sietzen is a deep learning engineer at Kaleido AI and a freelance video creator with an interest in video computing. Sietzen's project is a 3D visualisation of a CNN showing all the layers. He has built a Python script that trains a CNN in TensorFlow; it exports a JSON file that the CNN visualised can read automatically. 

Step 1: Loading TensorFlow checkpoints to build the graph

The researcher is loading and visualising Tensorflow weights and feature map activations from a custom JSON format. This has to be converted with a python script from original .ckpt binary files.

View a demonstration video here.

Step 2: Building the CNN in the Unity Editor

The next step is designing a CNN in the Unity Editor by combining pre-configured Game Objects (Prefabs) to build the graph. A large number of parameters in this mode allows the user to collapse and expand each layer differently. In addition, this makes it possible to seamlessly transition between showing filters as edges or showing all the calculations.

Edge bundled layers with one convolutional layer displaying the calculations.

Zoomed in calculation modes.

Fully connected layer displaying all weights.

Fully connected layer where only one edge leaves each input feature map.

Default view : Displaying the real number of filter weights as edges.

Smaller feature maps in full number.

Houdini 

Keras Model Learning

Keras is an easy-to-use Python library for developing and evaluating deep learning models. It is built on top of TensorFlow 2 and can scale to large clusters of GPUs, or an entire TPU pod, making it easy to create neural networks using the software. 

In the following demonstration, Himanshu Sharma, a data science master, has used Keras’ deep replay for the neural network training process.

Source: Himanshu Sharma111

Producing a Latex Code in Plot NeuralNet

The Python library, PlotNeuralNet by Haris Iqbal, is a Python package to generate a LaTeX code for drawing neural network models. Haris Iqbal holds a master in computer science and is working as a computer vision engineer at NavInfo. 

Unlike softwares that use architecture code, PlotNeuralNet works by separating and defining the network code for the program to consume. The program uses this to output the final diagram.  

CNN architecture: PlotNeuralNet (Github page)

 Holistically-Nested Edge Detection (view on Overleaf)

FCN-32 (view on Overleaf)

Source: Haris Iqbal on GitHub

Illustrations like these help in visualising how a neural network really looks and works, and the different elements that go with it. But, as we noted earlier, there is still plenty left to learn about deep learning. 

Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

cnn · 5 stories
  1. Top Research Papers On Recurrent Neural Networks For NLP Enthusiasts
  2. Now CNNs Can Work 10 Times Better With EfficientNet
  3. How ML Is Changing The Way We Use Touchscreens
  4. You are here: What A Neural Network Really Looks Like
  5. Are Deep Neural Networks Unequivocally Better Than Lidar?
About the author

Contributor

Avi Gopani is a technology journalist that seeks to analyse industry trends and developments from an interdisciplinary perspective at AIM. Her articles chronicle cultural, political and social stories that are curated with a focus on the evolving technologies of artificial intelligence and data analytics.

All stories
Belamy · Daily

The day’s AI news, before your first meeting.