Implicit Neural Representations yield memory-efficient shape or object or appearance or scene reconstructions for various machine learning problems, including 2D/3D images, videos, audio and wave problems. However, present implicit neural representations employ non-periodic activation functions such as ReLU, tanh, sigmoid and softplus. ReLU is linear, continuous and differentiable to first-order, but it cannot be differentiated twice. On the other hand, a few variants of ReLU, tanh, sigmoid and softplus are twice-differentiable and continuous. But, these functions are unable to handle a physical signal’s spatial and temporal derivatives. Therefore, these functions fail to yield satisfactory reconstructions for complex and higher-order problems.
Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell and Gordon Wetzstein of Stanford University have introduced a periodic activation network for representing higher-order complex problems. This periodic activation network produces the sine form of the input signal and is named Sinusoidal Representation Networks, shortly, SIREN. Because sinusoidal functions are differentiable to any degree, they help achieve precise 2D and 3D reconstructions along with their spatial and temporal derivatives. SIRENs are trained and validated for these representations using hyper networks with quick and accurate sine activation functions.
The SIREN helps modeling complex first-order and second-order ordinary differential equations (ODE) and partial differential equations (PDE), and solves them to great accuracy. The complex problems that the SIREN can solve include famous boundary value problems (BVP) and initial value problems (IVP) such as the Eikonal equations, the Poisson’s equations, the Helmholtz equation, the wave equation and the heat equation.
Python Implementation of SIREN
!git clone https://github.com/vsitzmann/siren.git
Download the Anaconda-3 package using the following command, if the local machine does not have a conda environment.
Install the downloaded Anaconda-3 package using the following command.
Enable the conda directory to run further commands,
%cd content/siren/ !export PATH=~/anaconda3/bin:$PATH !exec bash
and activate the environment by providing the following commands inside the inner base mode command cell as shown below.
Activate the model,
conda activate siren
The following command performs experimental image training.
!python experiment_scripts/train_img.py --model_type=sine
The following command performs experimental audio training on in-built audio clips.
!python experiment_scipts/train_audio.py --model_type=sine --wav_path=<path_to_audio_file>
The following command performs experimental video training on in-built bikes-video dataset.
!python experiment_scipts/train_video.py --model_type=sine --experiment_name bikes_video
The following command performs experimental 3D-scene reconstruction on in-built Thai statue data by fitting a signed distance function (SDF).
!python experiments_scripts/train_single_sdf.py --model_type=sine --point_cloud_path=<path_to_the_model_in_xyz_format> --batch_size=250000 --experiment_name=experiment_1
Apart from performing implicit neural representations on 2D/3D image, scene, video, audio datasets, the SIREN architecture is capable of solving complex higher-order differential equations, both ODE and PDE such as the wave equation, the heat equation, the Poisson’s equation, the Helmholtz equation and the Eikonal equations. SIREN’s periodic-activation-function approach may open a vast mathematical field of solving non-homogeneous and complex higher-order problems in the future.
- Original research paper
- Official website of SIREN
- Source code Repository
- Simplified PyTorch implementation
- SIREN in TensorFlow Playground
Join Our Telegram Group. Be part of an engaging online community. Join Here.
Subscribe to our NewsletterGet the latest updates and relevant offers by sharing your email.
A geek in Machine Learning with a Master's degree in Engineering and a passion for writing and exploring new things. Loves reading novels, cooking, practicing martial arts, and occasionally writing novels and poems.