Speech recognition with brain-computer interfaces
In an embodiment, a system includes a first sensor sensing brain signal data from a user, the brain signal data including nerve signals transmitted via a brain of the user and corresponding to a first set of words spoken by the user. The system also includes a second sensor sensing audio data from the user corresponding to the first set of words and one or more processors communicatively coupled to the first sensor and the second sensor. In the embodiment, the one or more processors generate text data based on the audio data using a machine learning algorithm and re-train the machine learning algorithm based on the brain signal data and the text data to generate a re-trained machine learning algorithm, wherein the re-trained machine learning algorithm generates second text data associated with a second set of words based on second brain signal data.
1. A system, comprising:
a first sensor configured to sense brain signal data from a user, the brain signal data comprising nerve signals transmitted via a brain of the user and corresponding to a first set of words spoken by the user;
a second sensor configured to sense audio data from the user corresponding to the first set of words;
one or more processors communicatively coupled to the first sensor and the second sensor, the one or more processors configured to:
generate text data based on the audio data using a machine learning algorithm; and
re-train the machine learning algorithm based on the brain signal data and the text data to generate a re-trained machine learning algorithm, wherein the re-trained machine learning algorithm is configured to generate second text data associated with a second set of words based on second brain signal data.
2. The system of claim 1 , comprising a wearable device configured to be worn on a body part of the user, wherein the wearable device comprises the second sensor.
3. The system of claim 1 , wherein the first sensor is disposed on the brain of the user.
4. The system of claim 1 , comprising a brain-computer interface configured to receive the brain signal data from the first sensor and extract features associated with the brain signal data.
5. The system of claim 4 , wherein:
the brain-computer interface comprises the one or more processors; and
the one or more processors are disposed on a wearable device configured to be worn on a body part of the user.
6. The system of claim 5 , wherein the wearable device comprises a frame and wherein the second sensor is disposed on the frame.
7. The system of claim 1 , wherein:
the audio data comprises a first time stamp and the brain signal data comprises a second time stamp; and
the one or more processors are configured to synchronize the brain signal data and the audio data according to the first time stamp and the second time stamp.
8. The system of claim 1 , wherein the first sensor is configured to sense the brain signal data and the second sensor is configured to sense the audio data concurrently.
9. A method, comprising:
receiving audio data and training brain signal data;
performing a comparison of a first time stamp and a first sensing duration associated with the audio data and a second time stamp and a second sensing duration associated with the training brain signal data;
generating time-synchronized data based on the comparison;
training a machine learning algorithm based on the time-synchronized data, wherein the machine learning algorithm is configured to generate text data based on brain signal data, wherein the brain signal data comprises electrical signals associated with the brain; and
receiving, from a sensor disposed on a brain of a user, the brain signal data, wherein the brain signal data corresponds to a set of words; and
generating, via the machine learning algorithm, the text data based on the brain signal data, wherein the text data corresponds to the set of words.
10. The method of claim 9 , comprising receiving, from the sensor, the training brain signal data, wherein the training brain signal data corresponds to a second set of words.
11. The method of claim 10 , comprising receiving, from a second sensor, the audio data, wherein the audio data corresponds to the second set of words.
12. The method of claim 11 , wherein the second sensor is a bone conduction microphone.
13. The method of claim 9 , wherein generating the time-synchronized data comprises synchronizing the audio data and the training brain signal data according to the first time stamp and the second time stamp.
14. The method of claim 13 , wherein synchronizing the audio data and the training brain signal data comprises generating a time delay to compensate for a lag of the training brain signal data behind the audio data.
15. The method of claim 14 , wherein synchronizing the audio data and the training brain signal data comprises applying the time delay to the audio data.
16. A device, comprising:
a sensor configured to sense brain signal data from a user, wherein the brain signal data comprises nerve signals transmitted via a brain of the user and the brain signal data corresponds to a first set of words;
one or more processors communicatively coupled to the sensor and configured to:
receive time-synchronized data comprising audio data and training brain signal data, wherein the time-synchronized data corresponds to a second set of words;
extract a first set of features associated with the audio data;
perform a comparison of the first set of features with a set of expected features corresponding to the second set of words;
generate training text data based on the comparison using speech recognition, wherein the training text data corresponds to the second set of words; and
train a machine learning algorithm based on the training brain signal data and the training text data, wherein the machine learning algorithm is configured to generate text data based on the brain signal data, wherein the text data comprises a text string corresponding to the first set of words; and
generate the text data based on the brain signal data.
17. The device of claim 16 , wherein the one or more processors are configured to:
extract a second set of features associated with the training brain signal data; and
associate the second set of features with the training text data, wherein a second set of expected features comprise the second set of features associated with the training brain signal data.
18. The device of claim 17 , wherein the one or more processors are configured to:
extract a third set of features associated with the brain signal data;
compare the third set of features with the second set of expected features; and
generate the text data based on comparing the third set of features with the second set of expected features.
19. The device of claim 16 , wherein the time synchronized data comprises a time stamp and a time duration associated with the audio data and the training brain signal data and wherein the first set of words are spoken by the user.
20. The device of claim 16 , wherein the first set of words are spoken by a second user.