AI-Based System Can Now Turn Brainwaves Into Text

AI-Based System Can Now Turn Brainwaves Into Tex

Scientists have now developed a new AI system that converts brain activity into text could result in transforming communication for people who can’t speak or type. The AI-based machine works by using brain implants to track what neurons do when someone is speaking something. According to experts, the system could eventually aid communication for patients who are unable to speak or type, such as those with locked-in syndrome.

Dr Joseph Makin, a co-author of the research from the University of California, San Francisco, said, “We are not there yet, but we think this could be the basis of a speech prosthesis.”


Sign up for your weekly dose of what's up in emerging technology.

How does the system work?

Explaining the process, the research stated that they recruited some participants with electrode array implants in their brain so that the researchers can monitor their epileptic seizures. Once these participants were read aloud from 50 set sentences multiple times, the team tracks the activity in their brain, also known as the neural activity. Then the data was then fed into a machine-learning algorithm that converted the brain activity data for each spoken sentence into a string of numbers, which was then fed into the next part of the system which turned it into a sequence of words.

According to the researchers, at first, the system was not perfect. Some of its mistakes were — sentences like “Those musicians harmonise marvellously” was decoded as “The spinach was a famous singer”, and sentences like “A roll of wire lay near the wall” became “Will robin wear a yellow lily”. However, gradually the team found the accuracy far higher than previous approaches. 

The researchers further added that the accuracy of the system varied from person to person. For one participant, the accuracy was just 3% of each sentence on average needed correcting, which was technically higher than the word error rate of 5% for professional human transcribers. However, fortunately, the algorithm only handles a small number of sentences.

Researchers also realised that training the algorithm on one participant’s data will mean that less training data was needed from the final user, which is going to make training less difficult for patients.

When asked an expert, Dr Christian Herff from Maastricht University, stated that the research was exciting because the intelligent machine used less than 40 minutes to train the data for each participant, and a limited collection of sentences, rather than the millions of hours typically needed. He said, “By doing so they achieve levels of accuracy that haven’t been achieved so far.” However, he also noted the AI-based system was not ready for many severely disabled patients as the machine relies on the brain activity when spoken out loud by the people.

More Great AIM Stories

Sejuti Das
Sejuti currently works as Associate Editor at Analytics India Magazine (AIM). Reach out at

Our Upcoming Events

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM