IP Library Granted Patent US 11,328,534
Granted Patent B2
US 11,328,534 · App. 17/016,144 · Granted May 10, 2022

Monitoring the performance of physical exercises

Inventors: Konstantin Mehl (Munich, DE); Maximilian Strobel (Munich, DE)
Assignee: KAIA HEALTH SOFTWARE GMBH
G06V40/23A61B5/1116A61B5/1118A61B5/1128A61B5/725A61B5/7405A63B24/0062G06T7/248G06T7/74A63B2024/0068A63B2220/806A63B2230/62G06T2207/10016G06T2207/20084G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,328,534
App. No.
17/016,144
Filed
Sep 9, 2020
Granted
May 10, 2022
Kind
B2
Art Unit
2666
USPC
382/107
Abstract

A method for monitoring a person performing a physical exercise based on a sequence of image frames showing an exercise activity of the person. The method includes extracting, based on the sequence of image frames, for each image frame a set of body key points using a neural network, the set of body key points being indicative of a posture of the person in the image frame; deriving, based on a subset of the body key points in each image frame, at least one characteristic parameter indicating a progression of a movement of the person; detecting a start loop condition by evaluating the time progression of the at least one characteristic parameter, said start loop condition indicating a transition from a start posture of the person to the movement of the person when performing the physical exercise, wherein a loop of exercising encompasses one single repetition of the physical exercise; detecting an end loop condition by evaluating the time progression of at least one of the characteristic parameters, said end loop condition indicating a transition from the movement of the person when performing the physical exercise to an intermediate posture, wherein, as a result, the start of the loop and the end of the loop are determined; and deriving the time period for a single loop of the physical exercise based on the start of the loop and the end of the loop and evaluating the time period.

Claims (16)

1. A method for monitoring a person performing a physical exercise based on a sequence of image frames showing an exercise activity of the person, the method comprising: extracting, based on the sequence of image frames, for each image frame a set of body key points using a neural network, the set of body key points being indicative of a posture of the person in the image frame; deriving, based on a subset of the body key points in each image frame, characteristic parameters indicating a progression of a movement of the person; detecting a start loop condition by evaluating a time progression of at least one of the characteristic parameters, said start loop condition indicating a transition from a start posture of the person to the movement of the person when performing the physical exercise, wherein a loop of exercising encompasses one single repetition of the physical exercise; detecting an end loop condition by evaluating the time progression of at least one of the characteristic parameters, said end loop condition indicating a transition from the movement of the person when performing the physical exercise to an intermediate posture, wherein, as a result, the start of the loop and the end of the loop are determined; and deriving a time period fora single loop of the physical exercise based on the start of the loop and the end of the loop and evaluating the time period.

2. The method according to claim 1 , wherein at least one of the characteristic parameters for a respective image frame is derived from coordinate values of the body key points of the respective image frame.

3. The method according to claim 1 , wherein for each of the image frames, at least one of the characteristic parameters is the Procrustes distance between the subset of body key points in a respective frame and the same subset of body key points in a reference frame.

4. The method according to claim 1 , further comprising detecting the start posture of the person by comparing the person's posture in at least one image frame of the sequence of image frames with at least one predefined criterion.

5. The method according to claim 1 , wherein an image frame in which the start loop condition is detected defines the start of the person's exercising activity.

6. The method according to claim 1 , wherein the start loop condition is detected at an image frame in which at least one of the characteristic parameters leaves a predetermined value range and changes with at least a minimum rate of change.

7. The method according to claim 1 , wherein detecting the start loop condition comprises detecting when at least one of the characteristic parameters leaves a predetermined value range corresponding to the person's start posture.

8. The method according to claim 1 , further comprising detecting at least one evaluation point in the person's movement by evaluating the time progression of at least one characteristic parameter indicating the person's movement.

9. The method according to claim 8 , further comprising evaluating the person's posture at the at least one evaluation point.

10. The method according to claim 1 , wherein evaluating the person's posture comprises comparing the person's posture with a set of predefined conditions.

11. The method according to claim 1 , wherein, based on the result of comparison between the person's posture and a set of predetermined feedback trigger conditions, feedback is provided to the person.

12. A mobile device comprising: a camera configured to capture a sequence of image frames showing an exercise activity of a person using the mobile device; and a controller configured to: extract a set of body key points using a neural network for each image frame among the sequence of image frames, the set of body key points being indicative of a posture of the person in each image frame, derive, based on a subset of the body key points in each image frame, characteristic parameters indicating a progression of a movement of the person, detect a start loop condition by evaluating a time progression of at least one of the characteristic parameters, said start loop condition indicating a transition from a start posture of the person to the movement of the person when performing the physical exercise, wherein a loop of exercising encompasses one single repetition of the physical exercise; detect an end loop condition by evaluating the time progression of at least one of the characteristic parameters, said end loop condition indicating a transition from the movement of the person when performing the physical exercise to an intermediate posture, wherein, as a result, the start of the loop and the end of the loop are determined, and derive a time period for a single loop of the physical exercise based on the start of the loop and the end of the loop and evaluate the time period.

13. A non-transitory computer storage readable medium comprising computer executable program code configured to perform the method according to claim 1 .

14. The method according to claim 1 , wherein at least one of the characteristic parameters is derived from a subset of body key points in the sequence of image frames using machine learning.

15. The method according to claim 2 , wherein at least one of the characteristic parameters is derived from a subset of body key points in the sequence of image frames using machine learning.

16. The method according to claim 3 , wherein at least one of the characteristic parameters is derived from a subset of body key points in the sequence of image frames using machine learning.

Assignments (2)
CHANGE OF ADDRESS Recorded Aug 10, 2022
From: KAIA HEALTH SOFTWARE GMBH
To: KAIA HEALTH SOFTWARE GMBH
Reel/Frame 061143/0734 →
CHANGE OF ADDRESS Recorded Jun 30, 2022
From: KAIA HEALTH SOFTWARE GMBH
To: KAIA HEALTH SOFTWARE GMBH
Reel/Frame 060557/0877 →
Priority Claims (1)
EP 18174657 · May 28, 2018 · regional
Continuity (2)
Continuation 16113507 · Aug 27, 2018
Related Publication 20200410220A1 · Dec 31, 2020
Cited By (1)
US 12,329,515