Systems and methods for improved speech recognition using neuromuscular information
View Patent ↗Systems and methods for using neuromuscular information to improve speech recognition. The system includes a plurality of neuromuscular sensors, arranged on one or more wearable devices, wherein the plurality of neuromuscular sensors is configured to continuously record a plurality of neuromuscular signals from a user, at least one storage device configured to store one or more trained statistical models, and at least one computer processor programmed to provide as an input to the one or more trained statistical models, the plurality of neuromuscular signals or signals derived from the plurality of neuromuscular signals, determine based, at least in part, on an output of the one or more trained statistical models, at least one instruction for modifying an operation of a speech recognizer, and provide the at least one instruction to the speech recognizer.
1. A computerized system for using neuromuscular information to improve speech recognition, the system comprising:
a plurality of neuromuscular sensors, arranged on one or more wearable devices, wherein the plurality of neuromuscular sensors is configured to continuously record a plurality of neuromuscular signals from a user;
at least one storage device configured to store one or more trained statistical models; and
at least one computer processor programmed to:
provide as an input to the one or more trained statistical models, the plurality of neuromuscular signals or signals derived from the plurality of neuromuscular signals;
determine based, at least in part, on an output of the one or more trained statistical models whether the user is holding a gesture, wherein determining whether the user is holding the gesture comprises evaluating the output at two or more time points to determine whether the user is performing the gesture at each of the two or more time points;
in response to determining that the user is holding the gesture:
generate a first instruction for modifying a state of a speech recognizer to a first one of an activated state, in which the speech recognizer is configured to perform speech recognition, and a deactivated state, in which the speech recognizer is configured to not perform speech recognition; and
provide the first instruction to the speech recognizer;
in response to determining that the user is not holding the gesture:
generate a second instruction for modifying the state of the speech recognizer to a second one of the activated state and the deactivated state; and
provide the second instruction to the speech recognizer.
2. The computerized system of claim 1 , wherein evaluating the output at two or more time points to determine whether the user is performing the gesture at each of the two or more time points comprises:
determining based, at least in part, on the output of the one or more trained statistical models, a musculo-skeletal representation of the user; and
determining, based on the determined musculo-skeletal representation, whether the user is performing the gesture at each of the two or more time points.
3. The computerized system of claim 1 , further comprising:
a communications interface configured to provide the first instruction and/or the second instruction from the at least one computer processor to the speech recognizer.
4. The computerized system of claim 1 , wherein generating the first instruction comprises generating the first instruction for modifying the state of the speech recognizer from the deactivated state to the activated state, and wherein generating the second instruction comprises generating the second instruction for modifying the state of the speech recognizer from the activated state to the deactivated state.
5. The computerized system of claim 1 , wherein generating the first instruction comprises generating the first instruction for modifying the state of the speech recognizer from the activated state to the deactivated state, and wherein generating the second instruction comprises generating the second instruction for modifying the state of the speech recognizer from the deactivated state to the activated state.
6. The computerized system of claim 2 , wherein the musculoskeletal representation comprises information relating to a movement, a force, a pose, a gesture, and/or a muscle activation.
7. A computer implemented method for using neuromuscular information to improve speech recognition, the method comprising:
providing, using at least one computer processor, as an input to one or more trained statistical models, a plurality of neuromuscular signals or signals derived from the plurality of neuromuscular signals, wherein the plurality of neuromuscular signals is received from a plurality of neuromuscular sensors arranged on one or more wearable devices, wherein the plurality of neuromuscular sensors is configured to continuously record the plurality of neuromuscular signals from a user;
determining, using the at least one computer processor, based, at least in part, on an output of the one or more trained statistical models whether the user is holding a gesture, wherein determining whether the user is holding the gesture comprises evaluating the output at two or more time points to determine whether the user is performing the gesture at each of the two or more time points;
in response to determining that the user is holding the gesture:
generating, using the at least one computer processor, a first instruction for modifying a state of a speech recognizer to a first one of an activated state, in which the speech recognizer is configured to perform speech recognition, and a deactivated state, in which the speech recognizer is configured to not perform speech recognition; and
providing, using the at least one computer processor, the first instruction to the speech recognizer;
in response to determining that the user is not holding the gesture:
generating, using the at least one computer processor, a second instruction for modifying the state of the speech recognizer to a second one of the activated state and the deactivated state; and
providing, using the at least one computer processor, the second instruction to the speech recognizer.
8. The method of claim 7 , wherein evaluating the output at two or more time points to determine whether the user is performing the gesture at each of the two or more time points comprises:
determining based, at least in part, on the output of the one or more trained statistical models, a musculo-skeletal representation of the user; and
determining, based on the determined musculo-skeletal representation, whether the user is performing the gesture at each of the two or more time points.
9. The method of claim 7 , further comprising:
providing, using a communications interface, the first instruction and/or the second instruction from the at least one computer processor to the speech recognizer.
10. The method of claim 7 , wherein generating the first instruction comprises generating the first instruction for modifying the state of the speech recognizer from the deactivated state to the activated state, and wherein generating the second instruction comprises generating the second instruction for modifying the state of the speech recognizer from the activated state to the deactivated state.
11. The method of claim 7 , wherein generating the first instruction comprises generating the first instruction for modifying the state of the speech recognizer from the activated state to the deactivated state, and wherein generating the second instruction comprises generating the second instruction for modifying the state of the speech recognizer from the deactivated state to the activated state.
12. The method of claim 8 , wherein the musculoskeletal representation comprises information relating to a movement, a force, a pose, a gesture, and/or a muscle activation.