Methods of decoding speech from the brain and systems for practicing the same
Provided are methods of decoding speech from the brain of a subject. The methods include detecting speech production signals from electrodes operably coupled to the speech motor cortex of a subject while the subject produces or imagines producing a speech sound. The methods further include deriving a speech production signal pattern from the detected speech production signals, and correlating the speech production signal pattern with a reference speech production signal pattern to decode speech from the brain of the subject. Speech communication systems and devices for practicing the subject methods are also provided.
1. A method of decoding speech from the brain of a subject, comprising:
detecting speech production signals from at least three electrodes directly in contact with the speech motor cortex of a subject while the subject produces or imagines producing a speech sound, wherein the speech motor cortex comprises ventral sensorimotor cortex;
deriving a speech production signal pattern from the detected speech production signals;
correlating the speech production signal pattern with a reference speech production signal pattern corresponding to features of speech associated with at least two speech articulators selected from the group consisting of lips, tongue, larynx, and jaw, to decode speech from the brain of the subject; and
producing the speech decoded from the brain of the subject, wherein the speech is produced in an audible form or in a text form.
2. The method of claim 1 , wherein the at least three electrodes comprises at least 10 electrodes.
3. The method of claim 1 , wherein the at least three electrodes comprises at least 20 electrodes.
4. The method of claim 1 , wherein the speech production signals are detected from a region of the ventral sensorimotor cortex selected from: the pre-central gyrus, the post-central gyms, and combinations thereof.
5. The method of claim 1 , wherein the at least three electrodes are directly in contact with the speech motor cortex by implantation on the surface of the speech motor cortex.
6. The method of claim 5 , wherein the speech production signals are detected using electrocorticography (ECoG).
7. The method of claim 1 , wherein the at least three electrodes are directly in contact with the speech motor cortex by insertion of the electrodes into the speech motor cortex.
8. The method of claim 1 , wherein detecting speech production signals comprises detecting local field potentials from the speech motor cortex using the at least three electrodes.
9. The method of claim 8 , wherein the speech production signals comprise the high-gamma frequency component (85-175 Hz) of the local field potentials.
10. The method of claim 1 , wherein the detecting comprises detecting a time-course of speech production signals from each of the least three electrodes.
11. The method of claim 10 , wherein the time-course of speech production signals is detected beginning from 250 milliseconds or more prior to the onset of the speech sound.
12. The method of claim 1 , wherein the speech sound is selected from the group consisting of: a phoneme, formant acoustics of a vowel, a diphone, a triphone, a consonant-vowel transition, a syllable, a word, a phrase, a sentence, and combinations thereof.
13. The method of claim 12 , wherein the speech sound is a syllable.
14. The method of claim 1 , wherein deriving a speech production signal pattern comprises performing time-frequency analysis of speech production signals detected from each of the at least three electrodes.
15. The method of claim 14 , wherein the time-frequency analysis is performed using a method selected from the group consisting of: Fast Fourier Transform (FFT), wavelet transform, Hilbert transform, and bandpass filtering.
16. The method of claim 1 , wherein the at least three electrodes are distributed across the speech motor cortex such that speech production signals corresponding to at least three speech articulators are detected.
17. The method of claim 16 , wherein the at least three speech articulators are selected from the group consisting of: the larynx, the lips, the jaw, and the tongue.
18. The method of claim 1 , wherein the subject has a speech impairment.
19. The method of claim 1 , wherein the method is carried out using a receiver unit, comprising:
a wireless receiver in communication with a wireless transmitter that receives the speech production signals detected from the at least three electrodes;
a speech generator;
a processor; and
a memory comprising instructions for execution by the processor for deriving the speech production signal pattern from the detected speech production signals, correlating the speech production signal pattern with the reference speech production signal pattern to decode the speech sound, and producing the speech sound using the speech generator.