Exercise feedback system for musculoskeletal exercises
An exercise feedback system receives exercise data captured by client devices of users performing musculoskeletal exercises. The exercise feedback system may provide captured images to a client device of a physical trainer (PT) who remotely provides feedback on the users' exercise performances, for example, by labeling images as indicative of proper or improper musculoskeletal form. A PT may track multiple users using a central feed, which includes content displayed in an order based on ranking of users by a model. Additionally, the exercise feedback system may provide an augmented reality (AR) environment. For instance, an AR graphic indicating a target musculoskeletal form for an exercise is overlaid on a video feed displayed by a client device. Responsive to detecting that a user's form is aligned to the AR graphic, the exercise feedback system may notify the user and trigger the start of the exercise.
1. A method comprising:
receiving a classification of musculoskeletal form input by each of a plurality of human providers;
determining a level of quality of classification for each of the plurality of human providers by comparing each classification to a known set of proper classifications of musculoskeletal form;
determining a subset of the plurality of human providers by filtering out at least one of the plurality of human providers having a level of quality of classification less than a threshold value;
selecting a human provider from the subset of the plurality of human providers;
receiving, by an exercise feedback system from a first client device of a user, a set of one or more images captured by a camera of the first client device, the set of images indicating a musculoskeletal form of the user while performing an exercise;
sending, by the exercise feedback system to a second client device of the selected human provider, at least one image of the set of images for display to the selected human provider;
receiving, by the exercise feedback system from the second client device, a classification of the musculoskeletal form of the user, wherein the classification is input by the selected human provider responsive to viewing the at least one image of the set of images displayed by the second client device;
generating feedback for the exercise using at least the classification; and
sending, by the exercise feedback system to the first client device, the feedback for display to the user.
2. The method of claim 1 , further comprising:
sending, the one or more images for display on the first client device;
receiving, from the first client device, a selection of the at least one image of the set of images to be sent to the second client device.
3. The method of claim 1 , further comprising:
receiving a first plurality of images captured by client devices of a first plurality of users including at least the user, each of the first plurality of images indicating musculoskeletal form of one of the first plurality of users while performing the exercise;
receiving a second plurality of images captured by client devices of a second plurality of users who did not perform the exercise, each of the second plurality of images indicating musculoskeletal form of one of the second plurality of users while not performing the exercise; and
determining, using the first plurality of images and the second plurality of images, that one or more health metrics of the first plurality of users improved relative to one or more health metrics of the second plurality of users.
4. The method of claim 3 , wherein determining that the one or more health metrics of the first plurality of users improved comprises:
determining a target musculoskeletal form for the exercise, the target musculoskeletal form indicating at least a target angle between a first segment and a second segment of a body of a given user while performing at least a portion of the exercise.
5. The method of claim 3 , further comprising:
determining the first plurality of users and the second plurality of users from a population of users of the exercise feedback system using at least one of demographic information or physiological information of the population of users.
6. The method of claim 1 , further comprising:
receiving a first set of classifications provided by a first set of physical trainers;
receiving a second set of classifications provided by a second set of physical trainers; and
generating scores for the first and second sets of physical trainers using the comparison of the first set of classifications and the second set of classifications, each score indicating a metric of one of the classifications provided by the corresponding physical trainer.
7. The method of claim 6 , further comprising:
receiving user information from a third client device of a new user of the exercise feedback system;
determining one or more candidate physical trainers of the first or second set of physical trainers using the scores; and
displaying, by the third client device to the new user, information describing the one or more candidate physical trainers.
8. The method of claim 7 , further comprising:
determining at least one preference of the new user using the user information, wherein the one or more candidate physical trainers is determined further by comparing the at least one preference with attributes of the first or second set of physical trainers.
9. The method of claim 1 , further comprising:
labeling, responsive to receiving the classification, an image of the at least one image of the set of images as indicative of proper or improper musculoskeletal form for the exercise; and
providing the feedback by displaying the labeled image on the first client device.
10. The method of claim 1 , wherein the classification indicates a score of the musculoskeletal form of the user, the feedback indicating a trend in scores for historical musculoskeletal forms of the user.
11. The method of claim 1 , wherein the camera and an electronic display are positioned on a same side of the first client device, and wherein the feedback is displayed on the electronic display while the user is performing the exercise.
12. The method of claim 1 , wherein the camera includes at least a depth sensor configured to capture 3D images.
13. The method of claim 1 , wherein the user is performing an exercise program including a plurality of repetitions of the exercise, the set of one or more images including images corresponding to different repetitions, the method further comprising:
providing, to the second client device, a duration of time that the user took to complete the plurality of repetitions of the exercise.
14. A non-transitory computer-readable storage medium storing instructions that when executed by a processor cause the processor to perform steps including:
receiving a classification of musculoskeletal form input by each of a plurality of human providers;
determining a level of quality of classification for each of the plurality of human providers by comparing each classification to a known set of proper classifications of musculoskeletal form;
determining a subset of the plurality of human providers by filtering out at least one of the plurality of human providers having a level of quality of classification less than a threshold value;
selecting a human provider from the subset of the plurality of human providers;
receiving, by an exercise feedback system from a first client device of a user, a set of one or more images captured by a camera of the first client device, the set of images indicating musculoskeletal form of the user while performing an exercise;
sending, by the exercise feedback system to a second client device of the selected human provider, at least one image of the set of images for display to the selected human provider;
receiving, by the exercise feedback system from the second client device, a classification of the musculoskeletal form of the user, wherein the classification is input by the selected human provider responsive to viewing the at least one image of the set of images displayed by the second client device;
generating feedback for the exercise using at least the classification; and
sending, by the exercise feedback system to the first client device, the feedback for display to the user.
15. The computer-readable storage medium of claim 14 , wherein the steps further comprise:
sending, the one or more images for display on the first client device;
receiving, from the first client device, a selection of the at least one image of the set of images to be sent to the second client device.
16. The computer-readable storage medium of claim 14 , wherein the steps further comprise:
receiving a first plurality of images captured by client devices of a first plurality of users including at least the user, each of the first plurality of images indicating musculoskeletal form of one of the first plurality of users while performing the exercise;
receiving a second plurality of images captured by client devices of a second plurality of users who did not perform the exercise, each of the second plurality of images indicating musculoskeletal form of one of the second plurality of users while not performing the exercise; and
determining, using the first plurality of images and the second plurality of images, that one or more health metrics of the first plurality of users improved relative to one or more health metrics of the second plurality of users.
17. The computer-readable storage medium of claim 16 , wherein determining that the one or more health metrics of the first plurality of users improved comprises:
determining a target musculoskeletal form for the exercise, the target musculoskeletal form indicating at least a target angle between a first segment and a second segment of a body of a given user while performing at least a portion of the exercise.
18. The computer-readable storage medium of claim 16 , wherein the steps further comprise:
determining the first plurality of users and the second plurality of users from a population of users of the exercise feedback system using at least one of demographic information or physiological information of the population of users.
19. The computer-readable storage medium of claim 14 , wherein the steps further comprise:
receiving a first set of classifications provided by a first set of physical trainers;
receiving a second set of classifications provided by a second set of physical trainers; and
generating scores for the first and second sets of physical trainers using the comparison of the first set of classifications and the second set of classifications, each score indicating a metric of one of the classifications provided by the corresponding physical trainer.
20. The computer-readable storage medium of claim 19 , wherein the steps further comprise:
receiving user information from a third client device of a new user of the exercise feedback system;
determining one or more candidate physical trainers of the first or second set of physical trainers using the scores; and
displaying, by the third client device to the new user, information describing the one or more candidate physical trainers.