Depth-based feature systems for classification applications
Human Computer Interfaces (HCI) may allow a user to interact with a computer via a variety of mechanisms, such as hand, head, and body gestures. Various of the disclosed embodiments allow information captured from a depth camera on an HCI system to be used to recognize such gestures. Particularly, the HCI system's depth sensor may capture depth frames of the user's movements over time. To discern gestures from these movements, the system may group portions of the user's anatomy represented by the depth data into classes. “Features” which reflect distinguishing features of the user's anatomy may be used to accomplish this classification. Some embodiments provide improved systems and methods for generating and/or selecting these features. Features prepared by various of the disclosed embodiments may be less susceptible to overfitting training data and may more quickly distinguish portions of the user's anatomy.
1. A computer system configured to recognize a user gesture from depth data, the computer system comprising:
at least one processor;
at least one memory comprising instructions configured to cause the computer system to perform a method comprising:
receiving depth data captured while a user performed a gesture;
applying a template to the depth data to generate a plurality of features; and
identifying the gesture based upon the plurality of features, wherein
the template was generated, at least in part, by:
determining a Gaussian function;
sampling the Gaussian function to generate a first distribution; and
generating a second distribution by iteratively sampling the first distribution and selecting points of the first distribution using a metric, wherein selecting points of the first distribution using a metric comprises determining a minimum of a plurality of Sub_Metric values, each Sub_Metric value determined using the formula
Sub_Metric
(
P
,
W
)
=
e
-
(
P
x
+
W
x
2
)
2
-
(
P
y
+
W
y
2
)
2
(
P
x
-
W
x
)
2
+
(
P
y
-
W
y
)
2
,
wherein P is a point from the first distribution,
wherein W is a point from the second distribution,
wherein P X is the X coordinate of the point P from the first distribution,
wherein P Y is the Y coordinate of the point P from the first distribution,
wherein W X is the X coordinate of the point W from the second distribution, and
wherein W Y is the Y coordinate of the point W from the second distribution.
2. The computer system of claim 1 , wherein the generated second distribution is a Well-Constructed Gaussian distribution and the template comprises offsets corresponding to the generated second distribution.
3. The computer system of claim 1 , wherein the template was generated by the computer system.
4. The computer system of claim 1 , wherein the template comprises a plurality of offsets corresponding to the second distribution and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest.
5. The computer system of claim 1 , the method further comprising:
receiving a plurality of training images; and
generating a plurality of trees from the training images and the template.
6. The computer system of claim 1 , wherein the generated second distribution is a Well-Constructed Gaussian distribution, the template comprises offsets corresponding to the generated second distribution, and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest generated using the template.
7. The computer system of claim 1 , wherein determining a minimum of a plurality of Sub_Metric values for points in the first distribution comprises determining a minimum of Sub_Metric values for a point in the first distribution and each existing point in the second distribution.
8. A computer-implemented method to recognize a user gesture from depth data, the method comprising:
receiving depth data captured while a user performed a gesture;
applying a template to the depth data to generate a plurality of features; and
identifying the gesture based upon the plurality of features, wherein
the template was generated, at least in part, by:
determining a Gaussian function;
sampling the Gaussian function to generate a first distribution; and
generating a second distribution by iteratively sampling the first distribution and selecting points of the first distribution using a metric, wherein selecting points of the first distribution using a metric comprises determining a minimum of a plurality of Sub_Metric values, each Sub_Metric value determined using the formula
Sub_Metric
(
P
,
W
)
=
e
-
(
P
x
+
W
x
2
)
2
-
(
P
y
+
W
y
2
)
2
(
P
x
-
W
x
)
2
+
(
P
y
-
W
y
)
2
,
wherein P is a point from the first distribution,
wherein W is a point from the second distribution,
wherein P X is the X coordinate of the point P from the first distribution,
wherein P Y is the Y coordinate of the point P from the first distribution,
wherein W X is the X coordinate of the point W from the second distribution, and
wherein W Y is the Y coordinate of the point W from the second distribution.
9. The computer-implemented method of claim 8 , wherein the generated second distribution is a Well-Constructed Gaussian distribution and the template comprises offsets corresponding to the generated second distribution.
10. The computer-implemented method of claim 8 , wherein the template was generated by the computer.
11. The computer-implemented method of claim 8 , wherein the template comprises a plurality of offsets corresponding to the second distribution and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest.
12. The computer-implemented method of claim 8 , the method further comprising:
receiving a plurality of training images; and
generating a plurality of trees from the training images and the template.
13. The computer-implemented method of claim 8 , wherein the generated second distribution is a Well-Constructed Gaussian distribution, the template comprises offsets corresponding to the generated second distribution, and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest generated using the template.
14. The computer-implemented method of claim 8 , wherein determining a minimum of a plurality of Sub_Metric values for points in the first distribution comprises determining a minimum of Sub_Metric values for a point in the first distribution and each existing point in the second distribution.
15. A non-transitory computer-readable medium comprising instructions configured to cause a computer system to perform a method to recognize a user gesture from depth data, the method comprising:
receiving depth data captured while a user performed a gesture;
applying a template to the depth data to generate a plurality of features; and
identifying the gesture based upon the plurality of features, wherein
the template was generated, at least in part, by:
determining a Gaussian function;
sampling the Gaussian function to generate a first distribution; and
generating a second distribution by iteratively sampling the first distribution and selecting points of the first distribution using a metric, wherein selecting points of the first distribution using a metric comprises determining a minimum of a plurality of Sub_Metric values, each Sub_Metric value determined using the formula
Sub_Metric
(
P
,
W
)
=
e
-
(
P
x
+
W
x
2
)
2
-
(
P
y
+
W
y
2
)
2
(
P
x
-
W
x
)
2
+
(
P
y
-
W
y
)
2
,
wherein P is a point from the first distribution,
wherein W is a point from the second distribution,
wherein P X is the X coordinate of the point P from the first distribution,
wherein P Y is the Y coordinate of the point P from the first distribution,
wherein W X is the X coordinate of the point W from the second distribution, and
wherein W Y is the Y coordinate of the point W from the second distribution.
16. The non-transitory computer-readable medium of claim 15 , wherein the generated second distribution is a Well-Constructed Gaussian distribution and the template comprises offsets corresponding to the generated second distribution.
17. The non-transitory computer-readable medium of claim 15 , wherein the template was generated by the computer system.
18. The non-transitory computer-readable medium of claim 15 , wherein the template comprises a plurality of offsets corresponding to the second distribution and wherein identifying the gesture based upon the plurality of features comprises applying the features to a forest.
19. The non-transitory computer-readable medium of claim 15 , the method further comprising:
receiving a plurality of training images; and
generating a plurality of trees from the training images and the template.
20. The non-transitory computer-readable medium of claim 15 , wherein determining a minimum of a plurality of Sub_Metric values for points in the first distribution comprises determining a minimum of Sub_Metric values for a point in the first distribution and each existing point in the second distribution.