“To our knowledge, this is the first successful demonstration of direct decoding of full words from the brain activity of someone who is paralyzed and cannot speak.”
Edward Chang, UCSF Neurosurgeon
Facebook Reality Labs (FRL) in collaboration with UCSF have developed a Brain Computer Interface (BCI) device that can accurately translate the brain signals to text. Facebook Reality Labs (FRL) was established in 2017 and the Brain-Computer Interface (BCI) project harboured an ambitious long-term goal: to develop a silent, non-invasive speech interface activated by thinking. As part of this, Project Steno was kicked off in 2019 in the Chang Lab at UCSF and involved a research participant who had lost the ability to speak following a series of strokes. The paralysed participant underwent elective surgery to plant electrodes on his brain.
To match speech patterns with the associated brain signals, the UCSF team collected dozens of hours of attempted speech with a BCI. The data collected through this process was then used to create machine learning models for speech detection and word classification. Project Steno is first of its kind to demonstrate a BCI that combines attempted speech with language models. “The result is a beautiful example of how we can leverage the statistical properties inherent in language — how one word can lead to another in the construction of a sentence — to dramatically increase the accuracy of a BCI,” said Emily Mugler, Neural Engineering Research Manager at FRL.
According to UCSF’s Dr.Edward Chang, the participant was able to build a 50-word library with the new technology and the efforts are underway to increase the capacity to 1,000 words, and eventually all words. “This is just the starting point,” said Chang.
“You actually have more of your brain dedicated to controlling your wrist than any other part of your body.”
Facebook Reality Labs
In spite of the success, Facebook said it’s distancing itself from projects like Steno. However, the company will open source their work on BCIs for the larger benefit of the community. Facebook clarified it has no interest in developing products that require implanted electrodes. Going forward, FRL will focus on wrist wear to scan brain signals. “We’ve decided to focus our immediate efforts on a different neural interface approach that has a nearer-term path to market: wrist-based devices powered by electromyography,” stated Facebook.
FRL’s wearables use electromyography (EMG) which translates nerve signals in the wrist to digital commands. For example,the QWERTY keyboard is over 150 years old, and it can be radically improved. Imagine a virtual keyboard that learns and adapts to your unique typing style (typos and all) over time.
The team FRL has open sourced its BCI software and have promised to share its head-mounted hardware prototypes with key researchers and peers to help advance new use cases.
The field of neuroprosthetics is in its nascence and it is too early to expect a widely adopted regulatory framework
It is easier to market wrist watches than electrodes plunged into the brain. While Elon Musk’s Neuralink is working on making brain implants become a reality, Facebook has decided to pursue non invasive and commercially viable products in the near future. Facebook’s U-turn on BCIs testifies that even a successful experiment might not convince a research lab to push further. Ethical concerns can put a damper on the researchers. The field of neuroprosthetics is in its nascence and it is too early to expect a widely adopted regulatory framework. Experts from Columbia University recommend establishing an equivalent of the hippocratic oath within the field– a “technocratic oath”, they call it.
In 2019, Columbia University’s Neurotechnology Center launched The NeuroRights Initiative (NRI) under the leadership of Dr. Rafael Yuste to serve as an advocacy organisation for human rights directives, and to develop further ethical guidance for neuro technological innovation.
The seven basic principles according to Dr. Yuste
- Have no intention of causing harm with the applied technology.
- Contribute to the common good with the work done
- Autonomy, which establishes that nothing can be done without the consent of those who are involved in any situation that involves AI and neurotechnology.
- Application of neurotechnology generates fair and impartial results, avoiding, for example, algorithmic biases.
- The integrity of the participants should be safeguarded.
- Elimination of all sensitive and identifiable information from the data collected by technology
- Algorithms used are as transparent and correctable as possible.
The potential of BCI based therapy for patients who had had stroke and other unfortunate accidents is immense. But, experts are urging for a framework or a bill to avoid future exploitations. For example, the South American nation Chile has recently approved bills to protect the right to personal identity, free will, mental privacy, equitable access to technologies that augment human capacities, and the right to protection against bias and discrimination.
