Active Hackathon

This Is How A Human Taught An AI To Beatbox Like A Pro

Of all the forms of music evolved organically, beatboxing is perhaps one of the most intriguing ones. It is the art of controlling one’s breath, lips, and mouth movement to create musical sounds that sound more like electronic beats. And while an artificial intelligence system may not possess any of these organs or body parts, it can now beatbox like a pro.

Second Self

To make the world understand and know what beatboxing really is, Nokia Bell Labs along with film production company Lonelyleap and award-winning beatboxer Harry Yeff a.k.a Reeps One came up with a documentary series — We Speak Music. This series not only focuses on the extraordinary human capabilities of the human voice but also focuses on the capability of the AI reaching that same level of beatboxing precision a human has.


Sign up for your weekly dose of what's up in emerging technology.

Talking about the series, Reeps One travels the world, learning about the potential of the human voice for creating sounds and connecting people. And finally ending up creating a spectacle performance using a sophisticated, deep learning AI system.

 “For centuries, humans have expressed creativity through their voices. These are what makes us unique and what artificial intelligence is starting to bring into question,” said Reeps One in the fifth episode of We Speak Music. “Now machines can be programmed to learn how to mimic the human voice. And begin to not just learn but create.”

The Experiment

In order to create the AI, Reeps One and CJ Carr worked on to run a deep learning experiment — neural network. They fed the machine with some sample beatbox audio by Reeps One (which acted as a dataset) and what they saw was the system was trying to figure out how to recreate the raw audio one sample at a time.

After a few thousand iterations, just like an embryo, the AI grew, and the AI was almost close to mimicking Reeps One’s beatboxing. And after 35,000+ iterations, the results that the AI showed was mind boggling — it was not only beatboxing, but it also caught the rhythm. But what was more amazing is the fact that this AI was even able to predict and also create the next part of the track.

In the sixth episode of We Speak Music, which is called the Second Self, Reeps One and his digital beatboxing A.I. ‘twin’ perform a duet. And it was done inside a chamber which was once the quietest room in the world, the Nokia Bell Labs Anechoic Chamber.

“AI could be compared to something like a nuclear bomb. There are two sides to it — one, space shuttles could be powered by this and on the other hand, it has the destructive power. It’s up to humans to be conscious and compassionate enough to do one or the other,” said CJ Carr, Programmer & Artist, Dadabots.

The Reeplicator AI

Another instance of AI making beatboxing look easy was when The Mill, a VFX and Creative Content studio, took its first foray into the world of Artificial Intelligence in 2018 by creating an AI called The Reeplicator AI, that could beatbox and that has performed live at SXSW.

In order to test the AI, The Mill collaborated the AI with Reeps One. The Mill decided to use a convolutional neural network to listen to Reeps One beatboxing and the AI could understand Reeps’ sound, his voice and even the beats he was making. To make things more advanced and to take a step further, they decided to train the system using machine learning by feeding different genres of tracks, so the AI could understand different type of types. And the results were outstanding, using its vocal sample bank built from Reeps’ voice the AI was able to create beats just like a human beatboxer.

This world’s first live beatboxing artificial intelligence, The Reeplicator AI when performed on stage, the audience experienced a beatbox battle between human (Reeps One) and machine (AI).


No doubt, beatboxing is not an easy form of music —the voice of a beatboxer is nothing less than an instrument. And documentaries like We Speak Music has shown how the world is starting to recognise this art of beatboxing.

But on the other hand, technology also seems to be on the race to become the best beatboxer. And projects like Second Self and The Reeplicator AI shows how the capabilities of AI to reach the human level. AI is definitely going all out to show the world that it can do wonder literally in any industry. But the major question here is — who has more to offer when it comes to beatboxing, AI or human?

More Great AIM Stories

Harshajit Sarmah
Harshajit is a writer / blogger / vlogger. A passionate music lover whose talents range from dance to video making to cooking. Football runs in his blood. Like literally! He is also a self-proclaimed technician and likes repairing and fixing stuff. When he is not writing or making videos, you can find him reading books/blogs or watching videos that motivate him or teaches him new things.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
How Data Science Can Help Overcome The Global Chip Shortage

China-Taiwan standoff might increase Global chip shortage

After Nancy Pelosi’s visit to Taiwan, Chinese aircraft are violating Taiwan’s airspace. The escalation made TSMC’s chairman go public and threaten the world with consequences. Can this move by China fuel a global chip shortage?

Another bill bites the dust

The Bill had faced heavy criticism from different stakeholders -citizens, tech firms, political parties since its inception

So long, Spotify

‘TikTok Music’ is set to take over the online streaming space, but there exists an app that has silently established itself in the Indian market.