Single brain implant gives paralyzed man bilingual communication

If things ultimately work out as hoped, brain implants will ultimately restore communication for those who have become paralyzed due to injury or disease. But we’re a long way from that future, and the implants are currently limited to testing in clinical trials.

One of those clinical trials, based at the University of California, San Francisco, has now inadvertently revealed something about how the brain handles language, because one of the patients enrolled in the trial was bilingual, using English and Spanish. By tracking activity in the area of the brain where the intention to speak gets translated into control over the vocal tract, researchers found that both languages produce consistent signals in this area, so training the system to pick up English phrases would help improve its recognition of Spanish.

Making some noise

Understanding bilingualism is obviously useful for understanding how the brain handles language in general. The new paper describing the work also points out that restoring communications in multiple languages should be a goal for restoring communications to people. Bilingual people will often change languages based on different social situations or sometimes do so within a sentence in order to express themselves more clearly. They often describe bilingual abilities as a key component of their personalities.

So, if we really want to restore communication to people, giving them access to all the languages they speak should be a part of it.

The new work is designed to make that more likely. Part of a clinical trial called BRAVO (brain-computer interface for restoration of arm and voice), it involved placing relatively simple implants (128 electrodes) into the motor region of the brain—the part that translates intentions to perform actions into the signals needed to trigger muscles to execute them.

In terms of speech, this means the neurons needed to convert the desire to say a word into the muscle activity needed to control the mouth and tongue, expel sufficient breath, and tension the vocal cords. This is downstream of the portion of the brain where word choices are made, and English and Spanish presumably differ (which is in turn downstream of where meaning is sorted out, where the two languages might overlap).

Hypothetically, if portions of words sound sufficiently similar in terms of how they sound, the muscle control needed to produce that sound would also be similar. So, by tracking neural activity here, they should be able to handle both languages and even detect overlaps between them.

Leveraging language

The process of detecting these signals is rather complex, given that neural activity looks like a noisy series of “spikes,” or bursts of voltage changes. Translating those to specific meanings is generally handled by AI systems that are trained to associate certain patterns of activity with specific information (whether that information is “I want to say cat” or “I’ve seen a cat”).

So, a lot of work here involved training the software portion of the system to recognize when the participant with the implant wanted to say specific words. This involved him imagining speaking them and the software knowing the word he intended to speak. Using this, the researchers trained the system to recognize 50 English words, 50 Spanish words, and a few that were identical in both languages. Because of things like verb tenses, this worked out to be 178 distinct words.