Identifying gestures using multiple sensors
View Patent ↗Systems and methods for recognizing human gestures are disclosed. In one embodiment, a method for recognizing a gesture made by an operator with a portable device, may comprise: obtaining a first sensor data profile associated with measurements made by the first sensor while the operator made a specific gesture involving the portable device; obtaining a second sensor data profile associated with measurements made by the second sensor while the operator made the specific gesture involving the portable device; and identifying the specific gesture by analyzing the first sensor data profile and the second sensor data profile.
1. A computer implemented method for recognizing a clapping gesture made by an operator with a portable device and generating an applause response, the portable device including at least a kinematic sensor and an audio sensor, the computer implemented method comprising:
obtaining a kinematic sensor data profile associated with measurements made by the kinematic sensor while the operator performs a specific gesture involving the portable device during a time window having a beginning and end;
wherein, the specific gesture includes the operator holding the portable device in a palm of a first hand and clapping the portable device into a palm of a second hand;
obtaining an audio sensor data profile associated with measurements made by the audio sensor while the operator performs the specific gesture involving the portable device;
wherein, the audio sensor data profile corresponds with a substantially identical time window as the kinematic sensor data profile;
analyzing the kinematic sensor data profile and the audio sensor data profile;
wherein the analyzing comprises:
normalizing the kinematic sensor data profile and audio sensor data profile;
comparing the normalized kinematic sensor data profile and normalized audio sensor data profile against a database of known sensor data profiles;
wherein, the database of known sensor data profiles comprises at least a first known sensor data profile known to correspond to a clapping gesture, the first known sensor data profile is characterized by an acceleration impulse near the beginning of the time window, a deceleration impulse near the end of the time window, and an audio signal impulse near the end of the time window;
determining that the specific gesture corresponds to a clapping gesture based on the analyzing of the kinematic sensor data profile and the audio sensor data profile; and
in response to determining that the specific gesture corresponds to a clapping gesture, generating an applause response;
wherein, the applause response comprises an audio output component; and
wherein, the audio output component of the applause response is based on the kinematic sensor data profile and the audio sensor data profile.
2. A computer implemented method as recited in claim 1 , further comprising: generating a gesture pattern for the operator according to multiple distinct inputs of gestures from the operator.
3. A computer implemented method as recited in claim 2 , wherein the gesture pattern is dynamically constructed or expanded by analyzing the operator's gesture pattern over time using predefined machine learning algorithms.
4. A computer implemented method as recited in claim 3 , further comprising: storing the operator's gesture pattern on the portable device and/or a cloud computing service.
5. A computer implemented method as recited in claim 1 , wherein the kinematic sensor is an accelerometer, and the audio sensor is a microphone.
6. A computer implemented method as recited in claim 1 , wherein the portable device further comprises a pressure sensor.
7. A computer implemented method as recited in claim 6 , the method further comprising:
obtaining a pressure sensor data profile associated with measurements made by the pressure sensor while the operator performs the specific gesture involving the portable device;
analyzing the pressure sensor data profile, wherein the analyzing comprises:
normalizing the pressure sensor data profile; and
comparing the normalized pressure sensor data profile against the database of known sensor data profiles;
wherein, the database of known sensor data profiles includes at least a second known sensor data profile known to correspond to a clapping gesture, the second known sensor data profile characterized by an acceleration impulse near the beginning of the time window, a deceleration impulse near the end of the time window, an audio signal impulse near the end of the time window, and a pressure impulse near the end of the time window; and
determining that the specific gesture corresponds to a clapping gesture based on the analyzing of the pressure sensor data profile, kinematic sensor data profile, and the audio sensor data profile.
8. A system for recognizing clapping gestures, the system comprising:
a portable device including at least a kinematic sensor and an audio sensor; and
a gesture recognition process executing on the system, the gesture recognition process operable to:
obtain a kinematic sensor data profile associated with measurements made by the kinematic sensor while an operator performs a specific gesture involving the portable device during a time window having a beginning and an end;
wherein, the specific gesture includes the operator holding the portable device in a palm of a first hand and clapping the portable device into a palm of a second hand;
obtain an audio sensor data profile associated with measurements made by the audio sensor while the operator performs the specific gesture involving the portable device;
wherein, the audio sensor data profile corresponds with a substantially identical time window as the kinematic sensor data profile;
analyze the kinematic sensor data profile and the audio sensor data profile, wherein the analysis comprises:
normalizing the kinematic sensor data profile and audio sensor data profile; and
comparing the normalized kinematic sensor data profile and normalized audio sensor data profile against a database of known sensor data profiles;
wherein, the database of known sensor data profiles includes at least a first known sensor data profile known to correspond to a clapping gesture, the first known sensor data profile characterized by an acceleration impulse near the beginning of the time window, a deceleration impulse near the end of the time window, and an audio signal impulse near the end of the time window;
determine that the specific gesture corresponds to a clapping gesture based on the analysis of the kinematic sensor data profile and the audio sensor data profile; and
in response to determining that the specific gesture corresponds to a clapping gesture, generate an applause response;
wherein the applause response comprises an audio output component;
wherein the audio output component of the applause response is based on the kinematic sensor data profile and the audio sensor data profile.
9. A system for recognizing clapping gestures as recited in claim 8 , wherein the gesture recognition process executes on the portable device.
10. A system for recognizing clapping gestures as recited in claim 8 , wherein the gesture recognition process is distributed across multiple computing devices and/or a cloud computing service.
11. A system for recognizing clapping gestures as recited in claim 8 , wherein the gesture recognition process is further operable to generate a gesture pattern for the operator according to multiple distinct inputs of gestures from the operator.
12. A system for recognizing clapping gestures as recited in claim 11 , wherein the gesture pattern is dynamically constructed or expanded by analyzing the operator's gesture pattern over time using predefined machine learning algorithms.
13. A system for recognizing clapping gestures as recited in claim 12 , wherein the gesture recognition process is further operable to store the operator's gesture pattern on the portable device, multiple computing devices, and a cloud computing service.
14. A system for recognizing clapping gestures as recited in claim 8 , wherein the kinematic sensor is an accelerometer, and the audio sensor is a microphone.
15. A system for recognizing clapping gestures as recited in claim 8 , wherein the portable device further comprises a pressure sensor.
16. A system for recognizing clapping gestures as recited in claim 15 , wherein the gesture recognition process is further operable to:
obtain a pressure sensor data profile associated with measurements made by the pressure sensor while the operator performs the specific gesture involving the portable device;
wherein, the pressure sensor profile corresponds with a substantially identical time window as the kinematic sensor data profile;
analyze the pressure sensor data profile, wherein the analysis comprises:
normalizing the pressure sensor data profile; and
comparing the normalized pressure sensor data profile against the database of known sensor data profiles;
wherein, the database of known sensor data profiles includes at least a second known sensor data profile known to correspond to a clapping gesture, the second known sensor data profile characterized by an acceleration impulse near the beginning of the time window, a deceleration impulse near the end of the time window, an audio signal impulse near the end of the time window, and a pressure impulse near the end of the time window; and
determine that the specific gesture corresponds to a clapping gesture based on the analysis of the pressure sensor data profile, kinematic sensor data profile, and the audio sensor data profile.
17. A system for recognizing clapping gestures as recited in claim 8 , wherein the system is configured to provide the operator a set of predefined functions of gesture recognition sensitivity according the operator's physical characteristics.