Systems and methods for monitoring and evaluating body movement
The present disclosure relates to systems and methods for analyzing and evaluating movement of a subjects and providing feedback. In some embodiments, a method comprises receiving one or more images of a body of the subject captured during performance of a physical movement by the subject; computing a model descriptive of positions and orientations of body parts of the subject based on the one or more images; generating a comparison of the positions and orientations to target positions and target orientations, respectively, for the physical movement; and generating a recommendation based on the comparison.
1. A method of evaluating movement of a subject, the method comprising:
receiving one or more images of a body of the subject captured during performance of a physical movement by the subject;
inputting the one or more images into an initial convolutional neural network (CNN) image classification model to identify subject positions from the received one or more images;
inputting the one or more images output from the initial CNN into a subsequent CNN image classification model in which the subsequent CNN image classification model is used to infer body pose estimations for the subject;
computing a subject model descriptive of positions and orientations of body parts of the subject based on the input of the one or more images to the subsequent CNN's image classification model;
generating a comparison of the positions and orientations to target positions and target orientations, respectively, for the physical movement;
generating a recommendation based on the comparison; and
exposing the recommendation on a client device for presenting to the subject.
2. The method of claim 1 , wherein the one or more images are in a form of video captured by the client device.
3. The method of claim 1 , wherein computing the subject model comprises:
identifying within the one or more images locations of joints associated with the body parts of the subject, in which the joints are identified based on the input of the one or more images to the subsequent CNN's image classification model; and
deriving the positions and orientations of the body parts from the identified locations of joints.
4. The method of claim 1 , wherein generating the comparison comprises, for each body part of the subject model:
computing positional deviations from the positions to the target positions;
computing orientational deviations from the orientations and the target orientations; and
computing the comparison as a score based on the computed positional deviations and orientational deviations.
5. The method of claim 4 , wherein generating the recommendation comprises:
generating, based on the score, an instruction to be provided to the subject that instructs the subject to perform a physical maneuver that, if so performed, will result in reduction of positional deviations and orientational deviations computed from a subsequently captured image of the body of the subject.
6. The method of claim 1 , further comprising:
generating or receiving authentication data indicative of an identity of the subject; and
storing the authentication data in a record associated with the subject.
7. The method of claim 6 , wherein generating the authentication data comprises:
generating the authentication data by identifying the subject within the one or more images.
8. A system for evaluating movement of a subject, the system comprising:
a memory; and
a processing device operatively coupled to the memory, wherein the processing device is configured to:
receive one or more images of a body of the subject captured during performance of a physical movement by the subject;
input the one or more images into an initial convolutional neural network (CNN) image classification model to identify subject positions from the received one or more images;
input the one or more images output from the initial CNN into a subsequent CNN image classification model in which the subsequent CNN image classification model is used to infer body pose estimations for the subject;
compute a subject model descriptive of positions and orientations of body parts of the subject based on the input of the one or more images to the subsequent CNN's image classification model;
generate a comparison of the positions and orientations to target positions and target orientations, respectively, for the physical movement;
generate a recommendation based on the comparison; and
expose the recommendation on a client device for presenting to the subject.
9. The system of claim 8 , wherein the one or more images are in a form of video captured by the client device.
10. The system of claim 8 , wherein to compute the subject model, the processing device is further configured to:
identify within the one or more images locations of joints associated with the body parts of the subject, in which the joints are identified based on the input of the one or more images to the subsequent CNN's image classification model; and
derive the positions and orientations of the body parts from the identified locations of joints.
11. The system of claim 8 , wherein to generate the comparison, the processing device is further configured to, for each body part of the subject model:
compute positional deviations from the positions to the target positions;
compute orientational deviations from the orientations and the target orientations; and
compute the comparison as a score based on the computed positional deviations and orientational deviations.
12. The system of claim 11 , wherein to generate the recommendation, the processing device is further configured to:
generate, based on the score, an instruction to be provided to the subject that instructs the subject to perform a physical maneuver that, if so performed, will result in reduction of positional deviations and orientational deviations computed from a subsequently captured image of the body of the subject.
13. The system of claim 8 , wherein the processing device is further configured to:
generate or receiving authentication data indicative of an identity of the subject; and
store the authentication data in a record associated with the subject.
14. The system of claim 13 , wherein to generate the authentication data, the processing device is further configured to:
generate the authentication data by identifying the subject within the one or more images.
15. A non-transitory, computer-readable storage medium having instructions encoded thereon that, when executed by a processing device of a client device, cause the processing device to:
receive one or more images of a body of a subject captured during performance of a physical movement by the subject;
inputting the one or more images into an initial convolutional neural network (CNN) image classification model to identify subject positions from the received one or more images;
input the one or more images output from the initial CNN into a subsequent CNN image classification model in which the subsequent CNN image classification model is used to infer body pose estimations for the subject;
compute a subject model descriptive of positions and orientations of body parts of the subject based on the input of the one or more images to the subsequent CNN's image classification model;
generate a comparison of the positions and orientations to target positions and target orientations, respectively, for the physical movement;
generate a recommendation based on the comparison; and
expose the recommendation on the client device for presenting to the subject.
16. The non-transitory, computer-readable storage medium of claim 15 , wherein the one or more images are in a form of video captured by the client device.
17. The non-transitory, computer-readable storage medium of claim 15 , wherein to compute the subject model, the instructions further cause the processing device to:
identify within the one or more images locations of joints associated with the body parts of the subject, in which the joints are identified based on the input of the one or more images to the subsequent CNN's image classification model; and
derive the positions and orientations of the body parts from the identified locations of joints.
18. The non-transitory, computer-readable storage medium of claim 15 , wherein to generate the comparison, the instructions further cause the processing device to, for each body part of the subject model:
compute positional deviations from the positions to the target positions;
compute orientational deviations from the orientations and the target orientations; and
compute the comparison as a score based on the computed positional deviations and orientational deviations.
19. The non-transitory, computer-readable storage medium of claim 18 , wherein to generate the recommendation, the instructions further cause the processing device to:
generate, based on the score, an instruction to be provided to the subject that instructs the subject to perform a physical maneuver that, if so performed, will result in reduction of positional deviations and orientational deviations computed from a subsequently captured image of the body of the subject.
20. The non-transitory, computer-readable storage medium of claim 15 , the instructions further cause the processing device to:
generate or receiving authentication data indicative of an identity of the subject; and
store the authentication data in a record associated with the subject.
21. The non-transitory, computer-readable storage medium of claim 20 , wherein to generate the authentication data, the instructions further cause the processing device to:
generate the authentication data by identifying the subject within the one or more images.
22. A non-transitory, computer-readable storage medium having instructions encoded thereon that, when executed by a processing device of a client device, cause the processing device to:
capture one or more images of a body of a subject during performance of a physical movement by the subject;
input the one or more images into an initial convolutional neural network (CNN) image classification model to identify subject positions from the received one or more images;
input the one or more images output from the initial CNN into a subsequent CNN image classification model in which the subsequent CNN image classification model is used to infer body pose estimations for the subject;
compute a subject model descriptive of positions and orientations of body parts of the subject based on the input of the one or more images to the subsequent CNN's image classification model;
generate a comparison of the positions and orientations to target positions and target orientations, respectively, for the physical movement;
generate a recommendation based on the comparison; and
present the recommendation to the subject.
23. The method of claim 1 , wherein the initial or subsequent CNN image classification model is optimized with hyperparameters to enhance the computed subject model.
24. The method of claim 1 , wherein the input of the one or more images to the initial CNN, the subsequent CNN, the computing of the subject model, and the generated recommendation is performed in real-time.
25. The method of claim 24 , wherein the real-time performance signifies the recommendation is generated during the performance of the physical movement by the subject.
26. The method of claim 1 , wherein the subject model is computed on the client device.
27. The method of claim 26 , wherein the client device is a smartphone computer.
28. The method of claim 1 , wherein the subsequent CNN is trained to identify positions and orientations of subjects.
29. The method of claim 28 , wherein the subsequent CNN is trained to locate focal points of the subject's body.
30. The method of claim 3 , further comprising rendering coordinate positions of the subject's joints based on the identified locations of joints.