What Are SIRENs and How Do They Work?

Implicit Neural Representations yield memory-efficient shape or object or appearance or scene reconstructions for various machine learning problems, including 2D/3D images, videos, audio and wave problems. However, present implicit neural representations employ non-periodic activation functions such as ReLU, tanh, sigmoid and softplus. ReLU is linear, continuous and differentiable to first-order, but it cannot be differentiated twice. On the other hand, a few variants of ReLU, tanh, sigmoid and softplus are twice-differentiable and continuous. But, these functions are unable to handle a physical signal’s spatial and temporal derivatives. Therefore, these functions fail to yield satisfactory reconstructions for complex and higher-order problems.

Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell and Gordon Wetzstein of Stanford University have introduced a periodic activation network for representing higher-order complex problems. This periodic activation network produces the sine form of the input signal and is named Sinusoidal Representation Networks, shortly, SIREN. Because sinusoidal functions are differentiable to any degree, they help achieve precise 2D and 3D reconstructions along with their spatial and temporal derivatives. SIRENs are trained and validated for these representations using hyper networks with quick and accurate sine activation functions.

Siren equation — The generalized form of the problems that the SIREN solves.

The SIREN helps modeling complex first-order and second-order ordinary differential equations (ODE) and partial differential equations (PDE), and solves them to great accuracy. The complex problems that the SIREN can solve include famous boundary value problems (BVP) and initial value problems (IVP) such as the Eikonal equations, the Poisson’s equations, the Helmholtz equation, the wave equation and the heat equation.

Siren evaluation — Comparison with other competing implicit neural representation architectures for 2D image reconstruction task in PSNR (Peak Signal-to-Noise Ratio) scale (Source).

Reconstruction of 2D image, its first derivative and second derivative by softplus, ReLU PE, tanh and SIREN architectures over several iterations (Source)

Siren on audio — Reconstruction of audio signal by ReLU, ReLU PE and SIREN architectures (Source)

Indoor video reconstruction by SIREN and ReLU architectures (Source)

Outdoor video reconstruction by SIREN and ReLU architectures (Source)

SIREN solves non-homogeneous Helmholtz equation over a few iterations while other functions fail at all (Source)

SIREN solves a wave equation over a few iterations while tanh function fails at all (Source)

Continuous 3D structure-aware Neural Scene Representation by SIREN: 3D reconstruction from a single-pose 2D image alone! (Source)

Python Implementation of SIREN

SIREN needs a GPU runtime, and PyTorch and conda environment. Download the source code into the local machine using the command,

!git clone https://github.com/vsitzmann/siren.git

Output:

Download the Anaconda-3 package using the following command, if the local machine does not have a conda environment.

!wget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh

Output:

Install the downloaded Anaconda-3 package using the following command.

!bash Anaconda3-2020.02-Linux-x86_64.sh

Enable the conda directory to run further commands,

 %cd content/siren/
 !export PATH=~/anaconda3/bin:$PATH
 !exec bash

and activate the environment by providing the following commands inside the inner base mode command cell as shown below.

Activate the model,

conda activate siren

The following command performs experimental image training.

!python experiment_scripts/train_img.py --model_type=sine

The following command performs experimental audio training on in-built audio clips.

!python experiment_scipts/train_audio.py --model_type=sine --wav_path=

The following command performs experimental video training on in-built bikes-video dataset.

!python experiment_scipts/train_video.py --model_type=sine --experiment_name bikes_video

The following command performs experimental 3D-scene reconstruction on in-built Thai statue data by fitting a signed distance function (SDF).

!python experiments_scripts/train_single_sdf.py --model_type=sine --point_cloud_path= --batch_size=250000 --experiment_name=experiment_1

Wrapping up

Apart from performing implicit neural representations on 2D/3D image, scene, video, audio datasets, the SIREN architecture is capable of solving complex higher-order differential equations, both ODE and PDE such as the wave equation, the heat equation, the Poisson’s equation, the Helmholtz equation and the Eikonal equations. SIREN’s periodic-activation-function approach may open a vast mathematical field of solving non-homogeneous and complex higher-order problems in the future.

Introduction to SIREN: Sinusoidal Representation Networks for Implicit Neural Representations

Introduction to SIREN: Sinusoidal Representation Networks for Implicit Neural Representations

Python Implementation of SIREN

Wrapping up

References:

Discussion

Advertise with Us

Our Events

AI Trainings

Wake up informed

Industry intelligence

Stay ahead with AI insights.