Customizing content for musculoskeletal exercise feedback
An exercise feedback system receives exercise data captured by client devices of users performing musculoskeletal exercises. The exercise feedback system may provide captured images to a client device of a physical trainer (PT) who remotely provides feedback on the users' exercise performances, for example, by labeling images as indicative of proper or improper musculoskeletal form. A PT may track multiple users using a central feed, which includes content displayed in an order based on ranking of users by a model. Additionally, the exercise feedback system may provide an augmented reality (AR) environment. For instance, an AR graphic indicating a target musculoskeletal form for an exercise is overlaid on a video feed displayed by a client device. Responsive to detecting that a user's form is aligned to the AR graphic, the exercise feedback system may notify the user and trigger the start of the exercise.
1. A method comprising:
receiving, by an exercise feedback system from a plurality of client devices, a plurality of images indicating musculoskeletal form of a plurality of users of the exercise feedback system while performing exercises;
determining user information describing performance of the exercises by the plurality of users;
generating, by a model receiving the plurality of images and the user information as input, a score for each of the plurality of users, wherein each score indicates a level of urgency that a corresponding user needs attention from a human provider, wherein the model generates each score by determining a weighted average of a plurality of signals including at least a level of pain for each of the plurality of users while performing the exercises;
sending, from the exercise feedback system to a client device of the human provider, at least a subset of the plurality of images, wherein the client device is different than the plurality of client devices;
displaying, by the client device to the human provider, the subset of the plurality of images in an order according to the scores of the plurality of users; and
receiving, by the exercise feedback system from the client device, feedback input by the human provider describing the musculoskeletal form of the plurality of users responsive to the human provider viewing the subset of the plurality of images displayed by the client device.
2. The method of claim 1 , wherein determining the user information comprises:
providing questions for display on the plurality of client devices periodically over a period of time; and
aggregating responses to the questions received from the plurality of client devices, the responses indicating at least one user-reported metric for the exercises.
3. The method of claim 2 , further comprising:
receiving, from a given client device of the plurality of client devices, a response to one of the questions at a timestamp; and
providing another one of the questions to the given client device at a time determined using the timestamp.
4. The method of claim 2 , wherein the at least one user-reported metric indicates the level of pain while performing the exercises or a level of difficulty of the exercises, and wherein aggregating the responses to the questions comprises:
calculating a moving average of the metric over the period of time, the model generating the scores using the moving average.
5. The method of claim 2 , wherein providing the questions for display comprises:
providing, to a given client device of a given user of the plurality of users, a plurality of instances of a same question for display over the period of time; and
wherein aggregating the responses to the questions comprises determining a trend of the given user over the period of time using responses to the same question, the trend indicating a change in the level of pain while performing the exercises or a level of difficulty of the exercises.
6. The method of claim 5 , further comprising:
determining one or more images of the plurality of images corresponding to the given user of the plurality of users; and
providing information describing the trend along with the one or more images to the client device for display.
7. The method of claim 1 , wherein determining the user information comprises:
determining, for each of the plurality of users, a timestamp of a most recently performed exercise by the user, the model generating the scores using the timestamps.
8. The method of claim 1 , wherein determining the user information comprises:
providing, to a given client device of a given user of the plurality of users, instructions for a test exercise;
receiving images from the given client device indicative of musculoskeletal form of the given user during performance of the test exercise; and
determining a metric of the performance of the test exercise by comparing the musculoskeletal form against a target musculoskeletal form for the test exercise, the model generating the score for the given user using the metric.
9. The method of claim 1 , wherein the human provider is a physical trainer associated with the plurality of users who uses the client device to input the feedback describing the musculoskeletal form of the plurality of users to the exercise feedback system.
10. The method of claim 1 , wherein the plurality of signals further includes a level of difficulty of the exercises and a timestamp of a most recently performed exercise for each of the plurality of users.
11. The method of claim 1 , further comprising:
receiving, as part of the feedback from the client device, one or more classifications of musculoskeletal form of the subset of the plurality of images; and
storing the one or more classifications along with corresponding images of the subset.
12. A non-transitory computer-readable storage medium storing instructions that when executed by a processor cause the processor to perform steps including:
receiving, by an exercise feedback system from a plurality of client devices, a plurality of images indicating musculoskeletal form of a plurality of users of the exercise feedback system while performing exercises;
determining user information describing performance of the exercises by the plurality of users;
generating, by a model receiving the plurality of images and the user information as input, a score for each of the plurality of users, wherein each score indicates a level of urgency that a corresponding user needs attention from a human provider, wherein the model generates each score by determining a weighted average of a plurality of signals including at least a level of pain for each of the plurality of users while performing the exercises;
sending, from the exercise feedback system to a client device of the human provider, at least a subset of the plurality of images, wherein the client device is different than the plurality of client devices;
displaying, by the client device to the human provider, the subset of the plurality of images in an order according to the scores of the plurality of users; and
receiving, by the exercise feedback system from the client device, feedback input by the human provider describing the musculoskeletal form of the plurality of users responsive to the human provider viewing the subset of the plurality of images displayed by the client device.
13. The computer-readable storage medium of claim 12 , wherein determining the user information comprises:
providing questions for display on the plurality of client devices periodically over a period of time; and
aggregating responses to the questions received from the plurality of client devices, the responses indicating at least one user-reported metric for the exercises.
14. The computer-readable storage medium of claim 13 , wherein the steps further comprise:
receiving, from a given client device of the plurality of client devices, a response to one of the questions at a timestamp; and
providing another one of the questions to the given client device at a time determined using the timestamp.
15. The computer-readable storage medium of claim 13 , wherein the at least one user-reported metric indicates the level of pain while performing the exercises or a level of difficulty of the exercises, and wherein aggregating the responses to the questions comprises:
calculating a moving average of the metric over the period of time, the model generating the scores using the moving average.
16. The computer-readable storage medium of claim 13 , wherein providing the questions for display comprises:
providing, to a given client device of a given user of the plurality of users, a plurality of instances of a same question for display over the period of time; and
wherein aggregating the responses to the questions comprises determining a trend of the given user over the period of time using responses to the same question, the trend indicating a change in the level of pain while performing the exercises or a level of difficulty of the exercises.
17. The computer-readable storage medium of claim 12 , wherein determining the user information comprises:
determining, for each of the plurality of users, a timestamp of a most recently performed exercise by the user, the model generating the scores using the timestamps.
18. The computer-readable storage medium of claim 12 , wherein determining the user information comprises:
providing, to a given client device of a given user of the plurality of users, instructions for a test exercise;
receiving images from the given client device indicative of musculoskeletal form of the given user during performance of the test exercise; and
determining a metric of the performance of the test exercise by comparing the musculoskeletal form against a target musculoskeletal form for the test exercise, the model generating the score for the given user using the metric.
19. The computer-readable storage medium of claim 12 , wherein the human provider is a physical trainer associated with the plurality of users who uses the client device to input the feedback describing the musculoskeletal form of the plurality of users to the exercise feedback system.
20. The computer-readable storage medium of claim 12 , wherein the plurality of signals further includes a level of difficulty of the exercises and a timestamp of a most recently performed exercise for each of the plurality of users.