Method and system for measuring and analyzing body movement, positioning and posture
One aspect of the invention provides a computer-based method for providing corrective feedback about exercise form, the method comprising; recording a user performing a specific exercise: evaluating, by the computer, with machine learning, computer vision, or deep learning models that have been previously trained in order to evaluate the form of a user by training on labelled and or unlabeled datasets that consist of: both correct and incorrect exercise form for the different types of exercises being evaluated; identifying the user throughout the video, the exercise type, each repetition of the exercise, the user's errors in form; and then generating, by the computer, corrective feedback for the user on how to improve exercise form for subsequent repetitions; and communicating, via an output device, the corrective feedback to the user.
1 . A method of generating user feedback from video of a user performing one or more reps of an exercise type, the method comprising:
receiving an exercise type;
selecting a rep detection model from a plurality of rep detection models, wherein the selected rep detection model corresponds to the exercise type;
selecting an error indication model from a plurality of error indication models, wherein the selected error indication model corresponds to the exercise type and comprises a machine learning algorithm trained on videos of individuals performing the exercise type;
receiving a video of a user performing one or more reps of the exercise type;
cropping the video to a specific aspect ratio;
identifying a plurality of frames in the video corresponding to a rep of the exercise type with the selected rep detection model;
generating a form error indication for the rep with the selected error indication model; and
generating feedback based on the form error indication;
wherein generating the form error indication for the detected rep with the selected error indication model comprises:
generating an error indication for each of the plurality of frames corresponding to the rep; and
generating the form error indication based at least in part on the error indication for at least one of the plurality of frames corresponding to the rep.
2 . The method according to claim 1 , wherein receiving the exercise type comprises receiving a user selection of one of a plurality of exercise types.
3 . The method according to claim 1 , wherein receiving the exercise type comprises generating an exercise type for the video with an exercise classification model.
4 . The method according to claim 1 , wherein receiving the video of the user performing one or more reps of the exercise type comprises capturing video of the user performing the reps with a video capture device.
5 . The method according to claim 1 , wherein the specific aspect ratio is one pixel by one pixel.
6 . The method according to claim 1 , wherein the error indication for each frame of the rep comprises an error probability, and generating the form error indication based at least in part on the error indication for at least one frame of the rep comprises generating a form error indication indicating the presence of an error if the error probability for at least one frame of the rep exceeds a probability threshold.
7 . The method according to claim 1 , wherein the error indication for each frame of the rep comprises an error probability, and generating the form error indication based at least in part on the error indication for at least one frame of the rep comprises generating a form error indication indicating the presence of an error if the error probability for a threshold number of frames of the rep exceed a probability threshold.
8 . The method according to claim 1 , wherein the error indication for each frame of the rep comprises an error probability, and generating the form error indication based at least in part on the error indication for at least one frame of the rep comprises generating a form error indication indicating the presence of an error if the error probability for a threshold number of consecutive frames of the rep exceed a probability threshold.
9 . The method according to claim 1 , wherein the error indication for each frame of the rep comprises an error probability, and generating the form error indication based at least in part on the error indication for at least one frame of the rep comprises:
calculating a sum of the error probabilities of the frames of the rep; and
generating a form error indication indicating the presence of an error if the sum of the error probabilities exceeds a probability threshold.
10 . The method according to claim 1 , wherein generating feedback based on the form error indication comprises selecting a media file corresponding to an error type of the error indication model if the form error indication indicates the presence of an error in the rep.
11 . The method according to claim 10 , wherein the media file comprises video of an individual demonstrating an error of the error type.
12 . The method according to claim 10 , wherein the media file comprises corrective audio instructions.
13 . The method according to claim 10 , wherein the media file comprises corrective video instructions.
14 . A non-transitory medium having stored thereon computer-readable instructions, which when executed by a data processor, cause the data processor to execute the method according to claim 5 .
15 . A method according to claim 5 , wherein the exercise type is one of: squats, deadlifts, pull-ups, push-ups, bench presses, planks, bicep curls, and a yoga pose.
16 . The method according to claim 1 , wherein the selected error indication model is configured to identify a body part of the user, and generating the form error indication for the detected rep with the selected error indication model further comprises:
identifying a position of the body part of the user in one of the plurality of frames; and
generating the form error indication based on the position of the body part of the user.
17 . The method according to claim 1 , wherein the selected error indication model is configured to identify an exercise equipment, and generating the form error indication for the detected rep with the selected error indication model further comprises:
identifying the exercise equipment in one of the plurality of frames; and
generating the form error indication based on the identified exercise equipment.
18 . A method of generating user feedback from video of a user performing one or more reps of an exercise type, the method comprising:
receiving an exercise type;
selecting a rep detection model from a plurality of rep detection models, wherein the selected rep detection model corresponds to the exercise type;
selecting an error indication model from a plurality of error indication models, wherein the selected error indication model corresponds to the exercise type and comprises a machine learning algorithm trained on videos of individuals performing the exercise type;
receiving a video of a user performing one or more reps of the exercise type;
adding pixels to the video to format the video to a specific aspect ratio;
identifying a plurality of frames in the video corresponding to a rep of the exercise type with the selected rep detection model;
generating a form error indication for the rep with the selected error indication model; and
generating feedback based on the form error indication;
wherein generating the form error indication for the detected rep with the selected error indication model comprises:
generating an error indication for each of the plurality of frames corresponding to the rep; and
generating the form error indication based at least in part on the error indication for at least one of the plurality of frames corresponding to the rep.
19 . A system for providing feedback to a user performing one or more reps of an exercise type, the system comprising:
a video capture device;
a processor;
a memory module;
an input module; and
an output module;
wherein the processor is configured to:
receive an exercise type;
capture video of the user performing one or more reps of the exercise type with the video capture device;
retrieve a rep detection model corresponding to the exercise type from the memory module;
crop the video to a specific aspect ratio;
execute the rep detection model to identify a plurality of frames in the video corresponding to a rep of the exercise type;
retrieve an error indication model corresponding to the exercise type from the memory module, wherein the error indication model comprises a machine learning algorithm trained on videos of individuals performing the exercise type;
execute the error indication model to generate a form error indication for the identified rep; and
output feedback with the output module based on the form error indication;
wherein the processor is configured to:
execute the error indication model to generate a form error indication for each of the plurality of frames corresponding to the rep; and
generate the form error indication based at least in part on the error indication for at least one of the plurality of frames corresponding to the rep.
20 . The system according to claim 19 , wherein the processor is configured to receive the exercise type from the input module.
21 . The system according to claim 19 , wherein the processor is configured to:
retrieve an exercise classification model from the memory module; and
execute the exercise classification model to generate the exercise type from the video.