Gesture cataloging and recognition
View Patent ↗Methods and apparatus for cataloging and recognizing gestures are disclosed. A gesture may be detected using sample motion data. An energy value and a baseline value may be computed. The baseline value may be updated if the energy value is below a calm energy threshold. The sample motion data may be adjusted based on the updated baseline value. A local variance may be calculated over a predetermined number of samples. Sample motion data values may be recorded if the local variance exceeds a threshold. Sample motion data recording may stop if a local variance scalar value falls below a drop threshold. Input Gestures may be recognized by computing a total variance for sample values in an Input Gesture; calculating a figure of merit using sample values from the Input Gesture and one or more Catalog Gestures; and determining whether the Input Gesture matches a Catalog Gesture from the figure of merit.
1. A method for gesture recognition, comprising:
a) receiving sample motion data from one or more sensors associated with a control device wherein the motion data is related to movement of the control device;
b) computing an energy value from the motion data and a baseline value for the motion data;
c) updating the baseline value based on the energy value if the energy value is less than a calm energy threshold;
d) adjusting the sample motion data based on the updated baseline value;
e) calculating a local variance of the sample motion data over a predetermined number of local variance samples;
f) beginning recording one or more values of the sample motion data for a gesture if the local variance scalar value is greater than a threshold for beginning recording;
g) calculating an average local variance scalar value using the one or more values of the sample motion data for a gesture; and
h) stopping the recording of the one or more values of the sample motion data for a gesture if the local variance scalar value is less than a threshold for stopping recording.
2. The method of claim 1 , further comprising:
i) computing a total variance for sample values in the gesture;
j) calculating a figure of merit using the sample values in the gesture and sample values in one or more catalog gesture, wherein the figure of merit is a measure of how well the gesture matched the catalog gesture;
k) determining whether an input gesture matches one of the one or more catalog gesture based on the figure of merit; and
l) taking action if the input gesture matches the one of the one or more catalog gesture.
3. The method of claim 2 , wherein the sample motion data comprises vector components.
4. The method of claim 3 , wherein the total variance is calculated with ranks assigned to components of the total variance corresponding to different ones of the vector components.
5. The method of claim 4 , wherein j) includes comparing input gesture component ranks to catalog gesture component ranks.
6. The method of claim 5 , wherein j) includes weighting components by ranking of total variance components.
7. The method of claim 6 , wherein j) includes calculating a correlation between the input gesture and the catalog gesture.
8. The method of claim 6 , further comprising scaling up the input gesture or catalog gesture based on which has fewer samples.
9. The method of claim 6 , further comprising comparing total variance rankings for input gesture and catalog gesture and applying a penalty for each of input gesture rank that does not match.
10. The method of claim 9 , wherein the figure of merit comprise a correlation scalar calculated as weighted average of scalar correlation with weights based on the total variance rankings.
11. The method of claim 10 , wherein total variance rankings are calculated at different time intervals within the gesture.
12. The method of claim 11 , wherein weights may be different at different intervals of time within the gesture.
13. The method of claim 2 , wherein the method is applied to a video game.
14. The method of claim 13 , wherein taking action comprises responding to command associated with the gesture.
15. The method of claim 14 , wherein comparing input gesture to selected subject of catalog gesture based on the game logic.
16. The method of claim 1 , wherein the sample motion data includes data derived from a signal from an inertial sensor.
17. The method of claim 1 , wherein the sample motion data includes data from an image from an image capturing device.
18. The method of claim 1 , wherein the sample motion data includes data derived from one or more signals obtained from one or more acoustic sensors.
19. An apparatus for gesture recognition comprising:
a control device;
a processor operable to execute a program of a method for gesture recognition, wherein the method comprising:
receiving sample motion data from one or more sensors associated with the control device wherein the motion data is related to movement of the control device;
computing an energy value from the motion data and a baseline value for the motion data;
updating the baseline value based on the energy value if the energy value is less than a calm energy threshold;
adjusting the sample motion data based on the updated baseline value;
calculating a local variance if the sample motion data over a predetermined number of local variance samples;
beginning recording one or more values of the sample motion data for a gesture if the local variance scalar value is greater than a start threshold;
calculating an average local variance scalar value using the one or more values of the sample motion data for a gesture; and
stopping the recording of the one or more values of the sample motion data for a gesture if the local variance scalar value is less than a stop threshold.
20. The apparatus of claim 19 , wherein the method for gesture recognition further comprises:
computing a total variance for sample values in the gesture;
calculating a figure of merit using the sample values in the gesture and sample values in one or more catalog gesture, wherein the figure of merit is a measure of how well the gesture matched the catalog gesture;
determining whether an input gesture matches one of the one or more catalog gesture based on the figure of merit; and
taking action if the input gesture matches the one of the one or more catalog gesture.
21. A non-transitory computer readable storage medium having computer readable instructions embodied therein, the computer readable instructions being configured to implement, when executed, a method for gesture recognition, the method comprising:
receiving sample motion data from a sensor associated with a control device wherein the motion data is related to movement of the control device;
computing an energy value from the motion data and a baseline value for the motion data;
updating the baseline value based on the energy value if the energy value is less than a calm energy threshold;
adjusting the sample motion data based on the updated baseline value;
calculating a local variance if the sample motion data over a predetermined number of local variance samples;
beginning recording one or more values of the sample motion data for a gesture if the local variance scalar value is greater than a start threshold;
calculating an average local variance scalar value using the one or more values of the sample motion data for a gesture; and
stopping the recording of the one or more values of the sample motion data for a gesture if the local variance scalar value is less than a stop threshold.