Gesture recognition system using depth perceptive sensors
View Patent ↗Acquired three-dimensional positional information is used to identify user created gesture(s), which gesture(s) are classified to determine appropriate input(s) to an associated electronic device or devices. Preferably at at least one instance of a time interval, the posture of a portion of a user is recognized, based at least one factor such as shape, position, orientation, velocity. Posture over each of the instance(s) is recognized as a combined gesture. Because acquired information is three-dimensional, two gestures may occur simultaneously.
1. One or more computer-storage devices having computer-executable instructions embodied thereon that when executed by a computing device perform a method of three-dimensional (“3D”) image analysis, the method comprising:
receiving 3D image data describing a 3D scene and comprising points having 3D coordinate information;
grouping at least some of the points into a plurality of clusters by using the points' depth values to form individual clusters comprising points that have similar depth values, wherein said grouping comprises testing adjacent points in the 3D image data for homogeneity using a gradient of the depth values for a cluster and splitting one or more regions and/or merging two or more adjacent regions, until no region needs be split and no two adjacent regions can be merged to form a cluster, wherein using the gradient of the depth values comprises determining that a region is homogeneous when a greatest gradient magnitude in the interior of the region is below a predefined threshold;
selecting, according to at least a first parameter, a specific cluster from the plurality of clusters, the specific cluster corresponding to a real-world object of interest described by the 3D image data;
grouping at least some of the points of the specific cluster into a set according to points' depth positions, wherein the set has a geometric center; and
associating a shape to the set, the shape being fixed to the geometric center of the set.
2. The devices of claim 1 , wherein the real-world object of interest is a person.
3. The devices of claim 1 , wherein the real-world object of interest is a person's body part.
4. The devices of claim 1 , wherein the method further comprises determining the geometric center for the set, the center having an assigned depth value that is an average of depth values assigned to points forming the set.
5. The devices of claim 1 , wherein the 3D image data is generated by a time-of-flight 3D camera.
6. One or more computer-storage devices having computer-executable instructions embodied thereon that when executed by a computing device perform a method of three-dimensional (“3D”) image analysis to identify the movement of objects in real time, the method comprising:
receiving 3D image data describing a scene and comprising points having 3 D coordinate information;
analyzing the 3D image data to identify a discrete gesture by:
(1) grouping at least some of the points into a plurality of clusters by using the points' depth values to form individual clusters comprising points that have similar depth values;
(2) selecting, according to at least a first parameter, a specific cluster from the plurality of clusters, the specific cluster corresponding to an object of interest;
(3) grouping at least some of the points of the specific cluster into a set according to points' positions in 3D space, wherein the set has a geometric center; and
(4) associating a 3D object to the at least one set, the 3D object being fixed to the geometric center of the at least one set; and
determining the beginning and end of a dynamic gesture through analysis of discrete gestures recognized at consecutive instances of time by using user-provided delimiter functions comprising a specific hand gesture, a specific sound, or a specific key input,
wherein a first user-provided delimiter function occurs prior to the dynamic gesture and a second user-provided delimiter function occurs after the dynamic gesture, and
wherein detection of the first user-provided delimiter function and detection of the second user-provided delimiter function indicates that the dynamic gesture occurred between the first user-provided delimiter function and the second user-provided delimiter function.
7. The devices of claim 6 , wherein the geometric center has a depth value equal to an average of depth values of points in the set.
8. The devices of claim 6 , wherein the 3D image data is generated by a time-of-flight 3D camera.
9. The devices of claim 6 , wherein the object of interest is a person.
10. The devices of claim 6 , wherein the object of interest is a person's body part.
11. The devices of claim 6 , wherein grouping at least some of the points into the plurality of clusters is based, at least partially, on a depth of a discrete region from a system used to capture the 3D image data.
12. The devices of claim 6 , wherein grouping at least some of the points into the plurality of clusters is based upon each point's z-depth value.
13. The devices of claim 6 , wherein the first parameter is the clusters' association with a common object.
14. A method of three-dimensional (“3D”) image analysis, the method comprising:
receiving 3D image data describing a scene and comprising points having 3D coordinate information;
grouping at least some of the points into a plurality of clusters by using the points' depth values to form individual clusters comprising points that have similar depth values;
selecting a specific cluster from the plurality of clusters, the specific cluster corresponding to a pre-identified object of interest, thereby providing a segmented image of the object of interest;
grouping at least some of the points of the specific cluster into a set according to points' depth positions, wherein the set has a geometric center;
associating a shape to the set, the shape being fixed to the geometric center of the set; and
determining the shape to the set by using a histogram comprising histogram values that are determined for randomly selected pixels of the segmented image.
15. The method of claim 14 , wherein the object of interest is a person.
16. The method of claim 14 , wherein the object of interest is a person's body part.
17. The method of claim 14 , wherein grouping at least some of the points into the plurality of clusters is based upon each point's z-depth value.
18. The method of claim 14 , wherein the method further comprises determining the geometric center for the set, the geometric center having an assigned depth value
that is an average of depth values assigned to the points forming the at least one set.
19. The method of claim 14 , wherein the method further comprises capturing the 3D image data using a time-of-flight 3D camera.
20. The method of claim 14 , wherein the specific cluster is selected according to the clusters' association with a common object.