Reservoir computing is an approach to make machine learning algorithms run faster. The word reservoir refers to a dynamical system.
A dynamical system is denoted by a mathematical function that describes how a point in space behaves with time. Having knowledge of these systems helps predict the future position of that point in space.
This reservoir consists of a bunch of recurrently connected units that are connected randomly.
In other words, reservoir computing uses a recurrent neural network, and instead of updating all parameters of the network, it only updates some of the parameters and keeps the other parameters fixed after choosing them randomly.
Reservoir computing (RC) is good at processing temporal or sequential kind of data. This is because of the way it is designed. The framework here resembles an RNN-based framework and the current RC frameworks are a product of echo state networks, liquid state machines and similar RNN models.
Many variants of RC models have been proposed to improve the performance of the original ones.
Here are a few different types of such reservoir computing:
- Context reverberation network
- Echo state network
- Backpropagation-decorrelation
- Liquid-state machine
- Nonlinear Transient Computation
- Deep reservoir computing
Reservoir Computing: But Why?
The role of the reservoir in RC is to transform the sequential inputs nonlinearly into a high-dimensional space so that the features of the inputs can be efficiently read out by a simple learning algorithm.
This makes it possible to use much faster learning algorithms. Instead of RNNs, other dynamical systems can also be used as reservoirs. This also led to the development of Physical Reservoir Computing.
“The simplicity of the training method in RC is attractive for non-expert developers;” said the researchers from the University of Tokyo in their recent work discussing in detail about the significance of reservoir computing.
The objective here is to establish systems that are capable of processing information faster at low learning cost and this becomes crucial in case of machine learning as the power consumption is usually high while training large datasets.
Brief Overview Of Physical RC
A conventional RC system has an RNN-based reservoir. Whereas, a physical RC system’s reservoir is realised using a physical system or device as shown above.
The extension of the reservoir computing framework towards Deep Learning was established with the introduction of Deep Reservoir Computing and of the Deep Echo State Network (DeepESN) model.
This also enabled the development of models that can be trained efficiently for hierarchical processing of temporal data, at the same time enabling the investigation on the inherent role of layered composition in recurrent neural networks.
The Physical Reservoir Computers(RC) are classified as follows:
- Fluidic Reservoir Computer
- Reservoir Computer using coupled oscillators
- Reservoir Computer using memristor
- Biological Reservoir Computer
These physical RC systems are useful for exploring the applicability of natural phenomena to information processing under realistic constraints in biology, chemistry, physics, and engineering.
The future development of reservoir computing is largely dependent on the innovations in material sciences. The applications of reservoir computing range from pattern recognition and time series forecasting to system approximation techniques in the machine learning context.
In case of biological reservoirs especially, the researchers at the University of Tokyo believe that new insights can be drawn through the mechanism of real-time information processing in a variety of brain regions. The researchers are now speculating on the implications of reservoir computing for brain interfaces. Parallels are being drawn between RC and biological functions. In case of the brain, some parts of the brain are likened to a reservoir just like what a memristor does in memristor-based RC.
