Recommendation engine for comparing physical activity to ground truth
A platform to accurately detect user pose/verify against a reference ground truth and provide feedback using an accuracy score that represents the deviation of the user pose from the reference ground truth, typically established by an expert.
1. A system comprising:
at least one processor configured to access at least one computer storage with instructions executable by the processor to:
obtain video images of a person undertaking a physical activity;
process the video images using at least one neural network to identify a type of the physical activity;
based at least in part on the type of physical activity, access at least one ground truth representation;
compare the physical activity undertaken by the person as captured in the video images to the ground truth representation; and
output a human-perceptible result of the comparison of the physical activity undertaken by the person as captured in the video images to the ground truth representation, wherein the at least one neural network is trained using a set comprising multiple coaches performing a set of poses, as well as non-experts performing the poses, to learn a similarity metric using a triplet loss imposing that a distance according to the metric is zero when two coaches are performing a same pose, that is as high as possible when two different poses are performed, and that, when non-experts are performing the poses, is higher than experts performing the poses as experts but lower than when performing a different pose, the similarity metric being used to evaluate pose correctness.
2. The system of claim 1 , wherein the instructions are executable to:
obtain the video images of the person undertaking the physical activity using real time simple optical capture of video images of the person.
3. The system of claim 2 , wherein the instructions are executable to identify the type of the physical activity without the person wearing passive or active image capture tags.
4. The system of claim 1 , wherein the instructions are executable to identify the physical activity as yoga.
5. The system of claim 1 , wherein the instructions are executable to identify the physical activity as golf.
6. The system of claim 1 , wherein the instructions are executable to obtain the video images from a fusion camera system comprising a red-green-blue (RGB) imager and at least one of a depth imager or a thermal imager or both a depth imager and a thermal imager.
7. The system of claim 1 , wherein the ground truth representation is stored in at least one database of images of experts executing ground truth activities.
8. The system of claim 7 , wherein the database provides friends recommendations and progress reports in similar endeavors.
9. The system of claim 1 , wherein the human-perceptible result of the comparison of the physical activity undertaken by the person as captured in the video images to the ground truth representation comprises a first image on a display of ground truth poses/motions from at least one expert and at least a second image of the user undertaking the physical activity.
10. The system of claim 1 , wherein the instructions are executable to output the human-perceptible result of the comparison in real time such that the person can view the result of the comparison simultaneously within processing time constraints with undertaking the physical activity.
11. A method comprising:
obtaining video images of a person undertaking a physical activity;
accessing at least one ground truth representation of the physical activity;
comparing information representing the physical activity undertaken by the person as captured in the video images to the ground truth representation; and
outputting a human-perceptible result of the comparison of the physical activity undertaken by the person as captured in the video images to the ground truth representation, wherein at least the comparing and the outputting comprise considering that (1) two different people having to perform a first pose differently, while the first pose is still considered equally correct for the purposes of the physical activity, and (2) some poses are more sensitive than other poses with respect to angular variations at some joints, sensitivity varying depending on variation.
12. The method of claim 11 , comprising:
obtaining the video images of the person undertaking the physical activity using real time simple optical capture of video images of the person.
13. The method of claim 12 , comprising identifying the type of the physical activity without the person wearing passive or active image capture tags.
14. The method of claim 11 , comprising identifying the physical activity as yoga.
15. The method of claim 11 , comprising identifying the physical activity as golf.
16. The method of claim 11 , comprising outputting the human-perceptible result of the comparison in real time such that the person can view the result of the comparison simultaneously within processing time constraints with undertaking the physical activity.