Wearable devices and methods for improved speech recognition
Systems and methods for using neuromuscular information to improve speech recognition. The system includes a plurality of neuromuscular sensors, arranged on one or more wearable devices, wherein the plurality of neuromuscular sensors is configured to continuously record a plurality of neuromuscular signals from a user, at least one storage device configured to store one or more trained statistical models, and at least one computer processor programmed to provide as an input to the one or more trained statistical models, the plurality of neuromuscular signals or signals derived from the plurality of neuromuscular signals, determine based, at least in part, on an output of the one or more trained statistical models, at least one instruction for modifying an operation of a speech recognizer, and provide the at least one instruction to the speech recognizer.
1. A wearable device, comprising:
one or more neuromuscular sensors configured to record a plurality of neuromuscular signals from a user donning the wearable device; and
one or more processors programmed to: provide the plurality of neuromuscular signals or signals derived from the plurality of neuromuscular signals as an input to one or more trained statistical models;
determine, based at least in part, on an output of the one or more trained statistical models, whether the user is holding a gesture, which includes evaluating two or more time points associated with the plurality of neuromuscular signals to determine whether the user is performing the gesture at each of the two or more time points;
generate a first instruction for modifying an operation of an active speech recognizer while the active speech recognizer k active in response to determining that the user is holding the gesture; and
generate a second instruction for modifying the operation of the active speech recognizer while the active speech recognizer is active in response to determining that the user is not holding the gesture.
2. The wearable device of claim 1 , further comprising at least one inertial measurement unit that is configured to record a movement of the user, wherein the input is a first input to the one or more trained statistical models and the one or more processors is further programmed to provide the movement of the user as a second input to the one or more trained statistical models.
3. The wearable device of claim 1 , wherein the gesture comprises contextual information associated with at least one of movements of the user or activation of muscles of the user.
4. The wearable device of claim 3 , wherein the speech recognizer interprets parts of speech provided to the speech recognizer by the user based on the contextual information.
5. The wearable device of claim 1 , wherein modifying the operation of the speech recognizer comprises changing an interaction mode of the speech recognizer.
6. The wearable device of claim 5 , wherein the interaction mode comprises at least one of a dictation mode, a spelling mode, an editing mode, or a navigation mode.
7. The wearable device of claim 1 , wherein the one or more processors is further programmed to output text in response to determining whether the user is holding the gesture.
8. The wearable device of claim 1 , wherein the one or more processors is further programmed to convert speech provided to the speech recognizer by the user to text.
9. The wearable device of claim 8 , wherein modifying the operation of the speech recognizer comprises instructing the speech recognizer to map the gesture to a linguistic token that is used to convert the speech to the text.
10. The wearable device of claim 8 , wherein the one or more processors is further programmed to correct the text converted from speech based on determining whether the user is holding the gesture.
11. A method comprising:
receiving a plurality of neuromuscular signals from a wearable device donned by a user;
providing the plurality of neuromuscular signals or signals derived from the plurality of neuromuscular signals as an input to one or more trained statistical models;
determining based at least in part, on an output of the one or more trained statistical models, whether the user is holding a gesture; which includes evaluating two or more time points associated with the plurality of neuromuscular signals to determine whether the user is performing the gesture at each of the two or more time points;
relaying a first instruction for modifying an operation of an active speech recognizer while the active speech recognizer is active in response to determining that the user k holding the gesture; and
relaying a second instruction for modifying the operation of the active speech recognizer while the active speech recognizer is active in response to determining that the user is not holding the gesture.
12. The method of claim 11 , wherein: the input is a first input to the one or more trained statistical models; and the method further comprises: configuring at least one inertial measurement unit of the wearable device to record a movement of the user; and providing data representative of the movement of the user as a second input to the one or more trained statistical models.
13. The method of claim 11 , wherein the gesture comprises contextual information associated with at least one of movements of the user or activation of muscles of the user.
14. The method of claim 13 , wherein the speech recognizer interprets parts of speech provided to the speech recognizer by the user based on the contextual information.
15. The method of claim 11 , wherein modifying the operation of the speech recognizer comprises changing an interaction mode of the speech recognizer.
16. The method of claim 15 , wherein the interaction mode comprises at least one of a dictation mode, a spelling mode, an editing mode, or a navigation mode.
17. The method of claim 11 , further comprising outputting text in response to determining whether the user is holding the gesture.
18. The method of claim 11 , further comprising converting speech provided to the speech recognizer by the user to text.
19. The method of claim 18 , wherein modifying the operation of the speech recognizer comprises instructing the speech recognizer to map the gesture to a linguistic token that is used to convert the speech to the text.
20. A non-transitory computer-readable medium encoded with instructions that, when executed by at least one computer processor performs a method of:
receiving a plurality of neuromuscular signals from a wearable device donned by a user;
providing the plurality of neuromuscular signals or signals derived from the plurality of neuromuscular signals as an input to one or more trained statistical models;
determining based at least in part, on an output of the one or more trained statistical models, whether the user is holding a gesture, which includes evaluating two or more time points associated with the plurality of neuromuscular signals to determine whether the user is performing the gesture at each of the two or more time points;
relaying a first instruction for modifying an operation of an active speech recognizer while the active speech recognizer is active in response to determining that the user is holding the gesture; and
relaying a second instruction for modifying the operation of the active speech recognizer while the active speech recognizer is active in response to determining that the user is not holding the gesture.