Systems, articles, and methods for gesture identification in wearable electromyography devices
View Patent ↗Systems, articles, and methods perform gesture identification with limited computational resources. A wearable electromyography (“EMG”) device includes multiple EMG sensors, an on-board processor, and a non-transitory processor-readable memory storing data and/or instructions for performing gesture identification. The wearable EMG device detects signals when a user performs a physical gesture and characterizes a signal vector {right arrow over (s)} based on features of the detected signals. A library of gesture template vectors G is stored in the memory of the wearable EMG device and a respective property of each respective angle θ i formed between the signal vector {right arrow over (s)} and respective ones of the gesture template vectors {right arrow over (g)} i is analyzed to match the direction of the signal vector {right arrow over (s)} to the direction of a particular gesture template vector {right arrow over (g)}*. The accuracy of the gesture identification may be enhanced by performing multiple iterations across multiple time-synchronized portions of the EMG signal data.
1. A method of operating a wearable electromyography (“EMG”) device, wherein the wearable EMG device includes a set of EMG sensors and a processor communicatively coupled to the set of EMG sensors, the method comprising:
detecting muscle activity of a user of the wearable EMG device by the set of EMG sensors, wherein the muscle activity corresponds to a user-performed gesture;
in response to detecting muscle activity of the user by the set of EMG sensors, providing a set of signals from the set of EMG sensors to the processor;
characterizing the set of signals as a first signal vector {right arrow over (s)} 1 by the processor;
determining a property of a first angle θ 1 formed between the first signal vector {right arrow over (s)} 1 and a first gesture template vector {right arrow over (g)} 1 by the processor, wherein the first gesture template vector {right arrow over (g)} 1 represents a first gesture; and
identifying the user-performed gesture by the processor, wherein identifying the user-performed gesture by the processor includes identifying the first gesture as the user-performed gesture by the processor if, at least, the property of the first angle θ 1 satisfies a criterion.
2. The method of claim 1 wherein characterizing the set of signals as a first signal vector {right arrow over (s)} 1 by the processor includes determining at least one feature of each signal in the set of signals by the processor, each at least one feature selected from the group consisting of: an average value of the signal, a mean value of the signal, a median value of the signal, a mode value of the signal, a maximum value of the signal, a minimum value of the signal, a standard deviation of the signal, a mean power frequency of the signal, and a root mean squared (“RMS”) value of the signal.
3. The method of claim 1 wherein determining a property of a first angle θ 1 formed between the first signal vector {right arrow over (s)} 1 and a first gesture template vector {right arrow over (g)} 1 by the processor includes determining, by the processor, at least one property selected from the group consisting of: a magnitude of the first angle θ 1 , a cosine of the first angle θ 1 , and a sine of the first angle θ 1 .
4. The method of claim 1 wherein the property of the first angle θ 1 does not satisfy the criterion, and wherein the method further comprises:
determining a property of a second angle θ 2 formed between the first signal vector {right arrow over (s)} 1 and a second gesture template vector {right arrow over (g)} 2 by the processor, wherein the second gesture template vector {right arrow over (g)} 2 represents a second gesture, and wherein identifying the user-performed gesture by the processor includes:
identifying the second gesture as the user-performed gesture by the processor if, at least, the property of the second angle θ 2 satisfies the criterion.
5. The method of claim 4 wherein the property of the second angle θ 2 does not satisfy the criterion, and wherein the method further comprises:
until an angle θ* having a property that satisfies the criterion is identified, iteratively:
determining a property of an angle θ i formed between the first signal vector {right arrow over (s)} 1 and an i th gesture template vector {right arrow over (g)} i by the processor, wherein i>2 and the i th gesture template vector {right arrow over (g)} i represents an i th gesture;
and wherein, in response to identifying an angle θ* having a property that satisfies the criterion, identifying the user-performed gesture by the processor includes:
stopping the iteration; and
identifying, by the processor, the gesture that is represented by a gesture template vector {right arrow over (g)}* corresponding to the angle θ* that satisfies the criterion as the user-performed gesture.
6. The method of claim 1 , further comprising:
for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, each gesture template vector {right arrow over (g)} i representing a respective gesture, determining a property of an angle θ i formed between the first signal vector {right arrow over (s)} 1 and the gesture template vector {right arrow over (g i )} by the processor, wherein determining a property of an angle θ i formed between the first signal vector {right arrow over (s)} 1 and the gesture template vector {right arrow over (g)} i by the processor for each gesture template vector {right arrow over (g)} i in the library of gesture template vectors G includes determining a property of a first angle θ 1 formed between the first signal vector {right arrow over (s)} 1 and a first gesture template vector {right arrow over (g)} 1 by the processor; and wherein:
identifying the user-performed gesture by the processor includes identifying, by the processor, a gesture represented by a gesture template vector {right arrow over (g)} i from the library of gesture template vectors G for which the property of the angle θ i satisfies a criterion, wherein identifying, by the processor, a gesture represented by a gesture template vector {right arrow over (g)} i from the library of gesture template vectors G for which the property of the angle θ i satisfies a criterion includes identifying the first gesture as the user-performed gesture by the processor if, at least: i) the property of the first angle θ 1 satisfies the criterion and ii) the property of the first angle θ 1 better satisfies the criterion than the property of any other angle θ i .
7. The method of claim 1 wherein the wearable EMG device further includes at least one inertial sensor, and wherein the method further comprises:
detecting motion of the wearable EMG device by the at least one inertial sensor, wherein the motion corresponds to the user-performed gesture;
in response to detecting motion of the wearable EMG device by the at least one inertial sensor, providing at least one signal from the at least one inertial sensor to the processor; and
processing the at least one signal from the at least one inertial sensor by the processor, and wherein identifying the user-performed gesture by the processor includes identifying the user-performed gesture by the processor based at least in part on an outcome of the processing the at least one signal from the at least one inertial sensor by the processor.
8. The method of claim 1 , further comprising:
capturing a respective time-synchronized first portion of each signal in the set of signals by the processor, wherein characterizing the set of signals as a first signal vector {right arrow over (s)} 1 by the processor includes characterizing the time-synchronized first portions of the signals in the set of signals as the first signal vector {right arrow over (s)} 1 by the processor;
capturing a respective time-synchronized second portion of each signal in the set of signals by the processor;
characterizing the time-synchronized second portions of the signals in the set of signals as a second signal vector {right arrow over (s)} 2 by the processor;
determining a property of a second angle φ 1 formed between the second signal vector {right arrow over (s)} 2 and the first gesture template vector {right arrow over (g)} 1 by the processor; and wherein:
identifying the user-performed gesture by the processor includes identifying the first gesture as the user-performed gesture by the processor if, at least: i) the property of the first angle θ 1 satisfies the criterion, and ii) the property of the second angle φ 1 satisfies the criterion.
9. The method of claim 1 wherein the wearable EMG device further includes a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores processor-executable gesture identification instructions, and wherein:
characterizing the set of signals as a first signal vector {right arrow over (s)} 1 by the processor includes executing, by the processor, a portion of the processor-executable gesture identification instructions to cause the processor to characterize the set of signals as a first signal vector {right arrow over (s)} 1 ;
determining a property of a first angle θ 1 formed between the first signal vector {right arrow over (s)} 1 and a first gesture template vector {right arrow over (g)} 1 by the processor includes executing, by the processor, a portion of the processor-executable gesture identification instructions to cause the processor to determine a property of a first angle θ 1 formed between the first signal vector {right arrow over (s)} 1 and a first gesture template vector {right arrow over (g)} 1 ; and
identifying the user-performed gesture by the processor includes executing, by the processor, a portion of the processor-executable gesture identification instructions to cause the processor to identify the user-performed gesture.
10. The method of claim 1 , further comprising:
in response to the user performing a reference gesture:
determining an orientation of the wearable EMG device on the user by the wearable EMG device; and
calibrating the first gesture template vector {right arrow over (g)} 1 by the wearable EMG device.
11. A wearable electromyography (“EMG”) device comprising:
a set of EMG sensors responsive to muscle activity of a user of the wearable EMG device, the muscle activity corresponding to a user-performed gesture, wherein in response to the muscle activity the set of EMG sensors provide a set of signals;
a processor communicatively coupled to the set of EMG sensors; and
a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores processor-executable gesture identification instructions that, when executed by the processor, cause the processor to:
characterize the set of signals as a first signal vector {right arrow over (s)} 1 ;
determine a property of a first angle θ 1 formed between the first signal vector {right arrow over (s)} 1 and a first gesture template vector {right arrow over (g)} 1 , wherein the first gesture template vector {right arrow over (g)} 1 represents a first gesture; and
identify the user-performed gesture, wherein identifying the user-performed gesture includes identifying the first gesture as the user-performed if, at least, the property of the first angle θ 1 satisfies a criterion.
12. The wearable EMG device of claim 11 , further comprising:
at least one communication terminal communicatively coupled to the processor, the at least one communication terminal to transmit information about the user-performed gesture to a receiving device.
13. The wearable EMG device of claim 11 , further comprising:
at least one inertial sensor communicatively coupled to the processor, the at least one inertial sensor responsive to motion corresponding to the user-performed gesture, wherein in response to motion the at least one inertial sensor provides at least one signal, and wherein the processor-executable gesture identification instructions that, when executed by the processor, cause the processor to identify the user-performed gesture cause the processor to identify the user-performed gesture based at least in part on the at least one signal provided by the at least one inertial sensor in response to the motion.
14. A method of operating a wearable electromyography (“EMG”) device, wherein the wearable EMG device includes a set of EMG sensors and a processor communicatively coupled to the set of EMG sensors, the method comprising:
detecting muscle activity of a user of the wearable EMG device by the set of EMG sensors, wherein the muscle activity corresponds to a user-performed gesture;
in response to detecting muscle activity of the user by the set of EMG sensors, providing a set of signals from the set of EMG sensors to the processor;
characterizing the set of signals as a first signal vector {right arrow over (s)} 1 by the processor;
for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, each gesture template vector {right arrow over (g)} i representing a respective gesture, determining, by the processor, a property of an angle θ i formed between the first signal vector {right arrow over (s)} 1 and the gesture template vector {right arrow over (g)} i ; and
identifying the user-performed gesture by the processor based at least in part on the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and a gesture template vector {right arrow over (g)} i .
15. The method of claim 14 wherein characterizing the set of signals as a first signal vector {right arrow over (s)} 1 by the processor includes determining at least one feature of each signal in the set of signals by the processor, each at least one feature selected from the group consisting of: an average value of the signal, a mean value of the signal, a median value of the signal, a mode value of the signal, a maximum value of the signal, a minimum value of the signal, a standard deviation of the signal, a mean power frequency of the signal, and a root mean squared (“RMS”) value of the signal.
16. The method of claim 14 wherein determining, for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, a property of an angle θ i formed between the first signal vector {right arrow over (s)} 1 and the gesture template vector {right arrow over (g)} i by the processor includes determining, by the processor and for each gesture template vector {right arrow over (g)} i in the library of gesture template vectors G, at least one property selected from the group consisting of: a magnitude of the angle θ i , a cosine of the angle θ i , and a sine of the angle θ i .
17. The method of claim 14 wherein determining, for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, a property of an angle θ i formed between the first signal vector {right arrow over (s)} 1 and the gesture template vector {right arrow over (g)} i by the processor includes determining a property of a first angle θ 1 formed between the first signal vector {right arrow over (s)} 1 and a first gesture template vector {right arrow over (g)} 1 by the processor, wherein the first gesture template vector {right arrow over (g)} 1 represents a first gesture, and wherein:
identifying the user-performed gesture by the processor based at least in part on the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and a gesture template vector {right arrow over (g)} i includes identifying the first gesture as the user-performed gesture by the processor if, at least, the property of the first angle θ 1 satisfies a criterion.
18. The method of claim 14 wherein the wearable EMG device further includes at least one inertial sensor, and wherein the method further comprises:
detecting motion of the wearable EMG device by the at least one inertial sensor, wherein the motion corresponds to the user-performed gesture;
in response to detecting motion of the wearable EMG device by the at least one inertial sensor, providing at least one signal from the at least one inertial sensor to the processor; and
processing the at least one signal from the at least one inertial sensor by the processor, and wherein identifying the user-performed gesture by the processor based at least in part on the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and a gesture template vector {right arrow over (g)} i includes identifying the user-performed gesture by the processor based at least in part on an outcome of the processing the at least one signal from the at least one inertial sensor by the processor.
19. The method of claim 14 , further comprising:
capturing a respective time-synchronized first portion of each signal in the set of signals by the processor, wherein characterizing the set of signals as a first signal vector {right arrow over (s)} 1 by the processor includes characterizing the time-synchronized first portions of the signals in the set of signals as the first signal vector {right arrow over (s)} 1 by the processor;
capturing a respective time-synchronized second portion of each signal in the set of signals by the processor;
characterizing the time-synchronized second portions of the signals in the set of signals as a second signal vector {right arrow over (s)} 2 by the processor; and
for each gesture template vector {right arrow over (g)} i in the library of gesture template vectors G, determining, by the processor, a property of an angle φ i formed between the second signal vector {right arrow over (s)} 2 and the gesture template vector {right arrow over (g)} i ; and wherein:
identifying the user-performed gesture by the processor based at least in part on the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and a gesture template vector {right arrow over (g)} i includes identifying the user-performed gesture by the processor based at least in part on both i) the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and a gesture template vector {right arrow over (g)} i , and ii) the property of at least one angle φ i formed between the second signal vector {right arrow over (s)} 2 and a gesture template vector {right arrow over (g)} i .
20. The method of claim 14 wherein the wearable EMG device further includes a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores processor-executable gesture identification instructions, and wherein:
characterizing the set of signals as a first signal vector {right arrow over (s)} 1 by the processor includes executing, by the processor, a portion of the processor-executable gesture identification instructions to cause the processor to characterize the set of signals as a first signal vector {right arrow over (s)} 1 ;
determining, by the processor, a property of an angle θ i formed between the first signal vector {right arrow over (s)} 1 and the gesture template vector {right arrow over (g)} i for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G includes executing, by the processor, a portion of the processor-executable gesture identification instructions to cause the processor to determine a property of an angle θ i formed between the first signal vector {right arrow over (s)} 1 and the gesture template vector {right arrow over (g)} i for each gesture template vector {right arrow over (g)} i in the library of gesture template vectors G; and
identifying the user-performed gesture by the processor based at least in part on the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and a gesture template vector {right arrow over (g)} i includes executing, by the processor, a portion of the processor-executable gesture identification instructions to cause the processor to identify the user-performed gesture by the processor based at least in part on the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and a gesture template vector {right arrow over (g)} i .
21. The method of claim 14 , further comprising:
in response to the user performing a reference gesture:
determining an orientation of the wearable EMG device on the user by the wearable EMG device; and
calibrating each gesture template vector {right arrow over (g)} i in the library of gesture template vectors G by the wearable EMG device.
22. A wearable electromyography (“EMG”) device comprising:
a set of EMG sensors responsive to muscle activity of a user of the wearable EMG device, the muscle activity corresponding to a user-performed gesture, wherein in response to the muscle activity the set of EMG sensors provide a set of signals;
a processor communicatively coupled to the set of EMG sensors; and
a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores processor-executable gesture identification instructions that, when executed by the processor, cause the processor to:
characterize the set of signals as a first signal vector {right arrow over (s)} 1 ;
for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, each gesture template vector {right arrow over (g)} i representing a respective gesture, determine a property of an angle θ i formed between the first signal vector {right arrow over (s)} 1 and the gesture template vector {right arrow over (g)} i ; and
identify the user-performed gesture based at least in part on the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and at least one gesture template vector {right arrow over (g)} i .
23. The wearable EMG device of claim 22 , further comprising:
at least one communication terminal communicatively coupled to the processor, the at least one communication terminal to transmit information about the user-performed gesture to a receiving device.
24. The wearable EMG device of claim 22 , further comprising:
at least one inertial sensor communicatively coupled to the processor, the at least one inertial sensor responsive to motion corresponding to the user-performed gesture, wherein in response to motion the at least one inertial sensor provides at least one signal, and wherein the processor-executable gesture identification instructions that, when executed by the processor, cause the processor to identify the user-performed gesture based at least in part on the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and at least one gesture template vector {right arrow over (g)} i cause the processor to identify the user-performed gesture based at least in part on both: i) the property of at least one angle θ i formed between the first signal vector {right arrow over (s)} 1 and at least one gesture template vector {right arrow over (g)} i , and ii) the at least one signal provided by the at least one inertial sensor in response to the detected motion.
25. A method of operating a wearable electromyography (“EMG”) device, wherein the wearable EMG device includes a set of EMG sensors and a processor communicatively coupled to the set of EMG sensors, the method comprising:
detecting muscle activity of a user of the wearable EMG device by the set of EMG sensors, wherein the muscle activity corresponds to a user-performed gesture;
in response to detecting muscle activity of the user by the set of EMG sensors, providing a set of signals from the set of EMG sensors to the processor;
until a same gesture is nominated as the user-performed gesture M times, where M≧2, iteratively:
capturing a respective time-synchronized portion of each signal in the set of signals by the processor;
characterizing the time-synchronized portions of the signals in the set of signals as a signal vector {right arrow over (s)} by the processor;
for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, each gesture template vector {right arrow over (g)} i representing a respective gesture, determining, by the processor, a property of an angle θ i formed between the signal vector {right arrow over (s)} and the gesture template vector {right arrow over (g)} i ; and
nominating, by the processor, a gesture as the user-performed gesture based on the determining, for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, a property of an angle θ i formed between the signal vector {right arrow over (s)} and the gesture template vector {right arrow over (g)} i ;
and
in response to nominating a same gesture as the user-performed gesture M times:
stopping the iteration; and
identifying, by the processor, the user-performed gesture as the gesture that is nominated as the user-performed gesture M times.
26. The method of claim 25 wherein characterizing the time-synchronized portions of the signals in the set of signals as a signal vector {right arrow over (s)} by the processor includes determining a respective feature of each respective time-synchronized portion of each signal in the set of signals by the processor, each respective feature selected from the group consisting of: an average value of the time-synchronized portion of the signal, a mean value of the time-synchronized portion of the signal, a median value of the time-synchronized portion of the signal, a mode value of the time-synchronized portion of the signal, a maximum value of the time-synchronized portion of the signal, a minimum value of the time-synchronized portion of the signal, a standard deviation of the time-synchronized portion of the signal, a mean power frequency of the time-synchronized portion of the signal, and a root mean squared (“RMS”) value of the time-synchronized portion of the signal.
27. The method of claim 25 wherein the wearable EMG device further includes at least one inertial sensor, and wherein the method further comprises:
detecting motion of the wearable EMG device by the at least one inertial sensor, wherein the motion corresponds to the user-performed gesture;
in response to detecting motion of the wearable EMG device by the at least one inertial sensor, providing at least one signal from the at least one inertial sensor to the processor;
processing the at least one signal from the at least one inertial sensor by the processor; and
identifying, by the processor, the user-performed gesture based at least in part on both: i) the gesture that is nominated as the user-performed gesture M times, and ii) an outcome of the processing the at least one signal from the at least one inertial sensor by the processor.
28. The method of claim 25 wherein for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, determining, by the processor, a property of an angle θ i formed between the signal vector {right arrow over (s)} and the gesture template vector {right arrow over (g)} i includes determining, by the processor, at least one property selected from the group consisting of: a magnitude of the angle θ i , a cosine of the angle θ i , and a sine of the angle θ i .
29. A wearable electromyography (“EMG”) device comprising:
a set of EMG sensors responsive to muscle activity of a user of the wearable EMG device, the muscle activity corresponding to a user-performed gesture, wherein in response to the muscle activity the set of EMG sensors provide a set of signals;
a processor communicatively coupled to the set of EMG sensors; and
a non-transitory processor-readable storage medium communicatively coupled to the processor, wherein the non-transitory processor-readable storage medium stores processor-executable gesture identification instructions that, when executed by the processor, cause the processor to:
until a same gesture is nominated as the user-performed gesture M times, where M≧2, iteratively:
capture a respective time-synchronized portion of each signal in the set of signals;
characterize the time-synchronized portions of the signals in the set of signals as a signal vector {right arrow over (s)};
for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, each gesture template vector {right arrow over (g)} i representing a respective gesture, determine a property of an angle θ i formed between the signal vector {right arrow over (s)} and the gesture template vector {right arrow over (g)} i ; and
nominate a gesture as the user-performed gesture based on determining, for each gesture template vector {right arrow over (g)} i in a library of gesture template vectors G, a property of an angle θ i formed between the signal vector {right arrow over (s)} and the gesture template vector {right arrow over (g)} i ; and
in response to nominating a same gesture as the user-performed gesture M times:
stop the iteration; and
identify the user-performed gesture as the gesture that is nominated as the user-performed gesture M times.
30. The wearable EMG device of claim 29 , further comprising:
at least one communication terminal communicatively coupled to the processor, the at least one communication terminal to transmit information about the user-performed gesture to a receiving device.
31. The wearable EMG device of claim 29 , further comprising:
at least one inertial sensor communicatively coupled to the processor, the at least one inertial sensor responsive to motion corresponding to the user-performed gesture, wherein in response to motion the at least one inertial sensor provides at least one signal, and wherein the processor-executable gesture identification instructions that, when executed by the processor, cause the processor to identify the user-performed gesture based at least in part on both: i) the gesture that is nominated as the user-performed gesture M times, and ii) an outcome of the processing the at least one signal from the at least one inertial sensor by the processor.