Nature Neuroscience· 2026Q1
Simultaneous speech and gesture decoding for multimodal communication in paralysis
- 1citations
- Q1SCImago
- 2026year
Short summary
A single high-density electrocorticography implant can simultaneously decode speech and gestures in individuals with paralysis, enabling control of a virtual avatar.
AI-generated from the title and abstract; the full text is not read.
Key points
- A single high-density electrocorticography implant can decode both speech and gestures simultaneously in participants with paralysis.
- Isolated upper-limb and orofacial movements were reliably decoded in three participants.
- Training decoders on both isolated and simultaneous speech/gesture data improved avatar control performance across contexts.
- This technology allows for multi-effector control from a single cortical implant.
AI-generated from the title and abstract; the full text is not read.
Abstract
Abstract Stroke and neurodegenerative diseases can impair speech and nonverbal gestures, limiting natural communication. Brain–computer interfaces (BCIs) aim to restore these functions by translating neural activity into commands for external devices, although prior work has primarily focused on decoding speech or gestures in isolation. Here we show that neural signals recorded with a single high-density electrocorticography implant can support simultaneous decoding of speech and gestures in people with paralysis. We first show that isolated upper-limb and orofacial movements can be reliably decoded among three participants. Using parallel speech and gesture decoders, we then enabled participants to control a personalized virtual avatar by attempting speech and gestures simultaneously or in isolation. Training models on both isolated and simultaneous data improved performance across behavioral contexts. These findings demonstrate that one cortical implant can support multi-effector control and provide a step toward BCIs that enable more natural communication for people with paralysis.
The authors' abstract, as published at the source. Nature Neuroscience, 2026 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Cognitive Neuroscience
Cognitive NeuroscienceNeuroscience