Fit-DNN: Deep neural networks using a single neuron

Folding-in-time concepts have already been successfully developed for related machine-learning methods called reservoir computing.

Researchers at Technical University, Berlin have proposed a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. The single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. By adjusting the feedback-modulation within the loops, the team adapted the network’s connection weights. The connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections are taken into account. The new method, Folded-in-time DNN (Fit-DNN), performed well in a set of benchmark tasks.

Folding-in-time concepts have already been successfully developed for related machine-learning methods called reservoir computing based on fixed recurrent neural networks. The one neuron concept will clearly benefit hardware implementation from a manufacturing standpoint. However, based on other results in the field of so-called time-delay reservoir computing, the researchers also believe their method could reduce the energy consumption of artificial neural networks.

If the time delay between two neurons “located” directly next to each other in time is shortened further, it would theoretically be possible to create a limitless number of neurons, the team posited.


Sign up for your weekly dose of what's up in emerging technology.

More Great AIM Stories

Kartik Wali
A writer by passion, Kartik strives to get a deep understanding of AI, Data analytics and its implementation on all walks of life. As a Senior Technology Journalist, Kartik looks forward to writing about the latest technological trends that transform the way of life!

AIM Upcoming Events

Regular Passes expire on 3rd Mar

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Early Bird Passes expire on 17th Feb

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, Virtual
Deep Learning DevCon 2023
27 May, 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

What went wrong with Meta?

Many users have opted out of Facebook and other applications tracking their activities now that they must explicitly ask for permission.