IP Library Granted Patent US 11,450,147
Granted Patent B2
US 11,450,147 · App. 16/735,101 · Granted Sep 20, 2022

Evaluation data structure for evaluating a specific motion pattern and dashboard for setting up the evaluation data structure

Inventors: Konstantin Mehl (Munich, DE); Maximilian Strobel (Munich, DE)
Assignee: KAIA HEALTH SOFTWARE GMBH
G06V40/23G06F16/51G06K9/6217G06K9/6267G06N5/003G06N20/20G06T7/20G06T7/60G06T7/70G06T2200/24G06T2207/20081G06T2207/30196G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,450,147
App. No.
16/735,101
Granted
Sep 20, 2022
Kind
B2
Abstract

An editor application configured for setting up at least one evaluation data structure is described, wherein each evaluation data structure is configured for evaluating a corresponding specific motion pattern in a sequence of image data structures. Each evaluation data structure comprises a ML model artifact configured for determining, based on input data comprising key data elements provided for at least one image data structure, class labels for each image data structure, said class labels identifying at least one of: at least one motion phase of the specific motion pattern, at least one evaluation point of the specific motion pattern. A key data element indicates a respective position of a landmark in the image data structure. Each evaluation data structure further comprises geometric evaluation data for performing a geometric evaluation of a configuration of key data elements of a particular image data structure that corresponds to an evaluation point or for performing a geometric evaluation of at least one motion phase of the specific motion pattern. Each evaluation data structure further comprises feedback data for providing a feedback to the user, said feedback depending on the result of the geometric evaluation. The editor application comprises at least one graphical user interface, the graphical user interface being configured for accepting user input for setting up and editing the geometric evaluation data and the feedback data.

Claims (59)

1. An editor application configured for setting up at least one evaluation data structure, each evaluation data structure being configured for evaluating a corresponding specific motion pattern in a sequence of image data structures, the specific motion pattern corresponding to a particular physical exercise, wherein each evaluation data structure comprises:

a machine learning (ML) model artifact of an exercise specific ML model configured for evaluating the particular physical exercise,

wherein the ML model is trained based on a plurality of sequences of image data structures showing different variants of the specific motion pattern for the particular physical exercise,

wherein ML model is configured to:

determine, based on input data comprising key data elements provided for at least one image data structure, class labels for each image data structure, said class labels identifying at least one of: at least one motion phase of the specific motion pattern and at least one evaluation point of the specific motion pattern, wherein the key data elements indicate positions of landmarks in the image data structures; and

generate geometric evaluation data and perform a geometric evaluation of a configuration of key data elements of a particular image data structure that corresponds to an evaluation point or for performing a geometric evaluation of at least one motion phase of the specific motion pattern; and

feedback data for providing a feedback to the user, said feedback depending on the result of the geometric evaluation,

wherein the editor application comprises at least one graphical user interface, the graphical user interface being configured for accepting user input for setting up and editing the geometric evaluation data and the feedback data, and

wherein at least one of the class labels identifies an evaluation point, the evaluation point being selected from the group comprising a start point of a motion phase, an end point of a motion phase, and an intermediate point in time of a motion phase where a reversal of a movement occurs, and wherein the geometric evaluation data comprises geometric evaluation data for performing a geometric evaluation of a configuration of key data elements of a particular image data structure that corresponds to said evaluation point.

2. The editor application according to claim 1 , wherein the editor application is configured for providing at least one dashboard, the at least one dashboard being configured for accepting user input for setting up and editing the geometric evaluation data and the feedback data of the evaluation data structure.

3. The editor application according to claim 1 , wherein the editor application is a web application, wherein the at least one graphical user interface of the editor application is accessible via the internet.

4. The editor application according to claim 1 , wherein training of the ML model underlying the ML model artifact is based on a plurality of sequences of image data structures showing different variants of the specific motion pattern, wherein for each image data structure, a set of key data elements is provided, the set of key data elements indicating a respective position of a landmark in the image data structure, said training being further based on class labels provided for each image data structure.

5. The editor application according to claim 1 , wherein the class labels are for the at least one motion phase and for the at least one evaluation point, and

wherein there is a predefined correlation between the at least one motion phase and the at least one evaluation point, with an evaluation point being a specific point of time within a motion phase or between consecutive motion phases.

6. The editor application according to claim 1 , wherein the ML model underlying the ML model artifact is a decision tree or a random forest comprising at least one decision tree.

7. The editor application according to claim 1 , wherein the geometric evaluation data is configured for evaluation of a representation of the person's body in an image data structure that corresponds to an evaluation point.

8. The editor application according to claim 1 , wherein the geometric evaluation data is configured for evaluation of a configuration of body key points of a particular pose in an image data structure that corresponds to an evaluation point.

9. The editor application according to claim 1 , wherein the geometric evaluation data comprises at least one geometric constraint.

10. The editor application according to claim 9 , wherein the at least one graphical user interface is configured for accepting user input for setting up and editing the at least one geometric constraint.

11. A method for setting up an evaluation data structure configured for evaluating a specific motion pattern in a sequence of image data structures, the specific motion pattern corresponding to a particular physical exercise, the method comprising:

providing a machine learning (ML) model artifact of an exercise specific ML model,

training the ML model based on a plurality of sequences of image data structures showing different variants of the specific motion pattern for the particular physical exercise,

evaluating, by the ML model, the particular physical exercise, including determining, by the ML model, based on input data comprising key data elements provided for at least one image data structure, class labels for each image data structure, said class labels identifying at least one of: at least one motion phase of the specific motion pattern and at least one evaluation point of the specific motion pattern, wherein the key data elements indicate positions of landmarks in the image data structures;

generating geometric evaluation data, by the ML model, and performing, by the ML model, a geometric evaluation of the configuration of the key data elements of a particular image data structure at one or more evaluation points or for performing a geometric evaluation of at least one motion phase of the specific motion pattern; and

specifying feedback data for providing feedback, the feedback depending on the result of the geometric evaluation,

wherein at least one of the class labels identifies an evaluation point, the evaluation point being selected from the group comprising a start point of a motion phase, an end point of a motion phase, and an intermediate point in time of a motion phase where a reversal of a movement occurs, and

wherein the geometric evaluation data comprises geometric evaluation data for performing a geometric evaluation of a configuration of key data elements of a particular image data structure that corresponds to said evaluation point.

12. The method according to claim 11 , further comprising a step of training the ML model, wherein the ML model underlies the ML model artifact, wherein said training is based on a plurality of sequences of image data structures showing different variants of the specific motion pattern, wherein for each image data structure, a set of key data elements is provided, said training being further based on the class labels provided for each image data structure.

13. The method according to claim 12 , wherein after the training step, the ML model artifact determines the class labels for the image data structures of the sequence of image data structures showing the specific motion pattern.

14. The method according to claim 12 , wherein after the training step, the ML model is configured for evaluating a specific motion pattern.

15. A system comprising

an editor application configured for setting up at least one evaluation data structure, each evaluation data structure being configured for evaluating a specific motion pattern in a sequence of image data structures, the specific motion pattern corresponding to a particular physical exercise;

a web server configured for storing the at least one evaluation data structure;

a mobile device configured for downloading at least one of the evaluation data structures from the web server and for using the at least one evaluation data structure for evaluating a specific motion pattern,

wherein the evaluation data structure comprises:

a machine learning (ML) model artifact of an exercise specific ML model configured for evaluating the particular physical exercise,

wherein the ML model is trained based on a plurality of sequences of image data structures showing different variants of the specific motion pattern for the particular physical exercise,

wherein the ML model is configured to:

determine, based on input data comprising key data elements provided for at least one image data structure, class labels for each image data structure, said class labels identifying at least one of: at least one motion phase of the specific motion pattern and at least one evaluation point of the specific motion pattern, wherein the key data elements indicate positions of landmarks in the image data structures; and

generate geometric evaluation data and perform a geometric evaluation of a configuration of key data elements of a particular image data structure that corresponds to an evaluation point or for performing a geometric evaluation of at least one motion phase of the specific motion pattern; and

feedback data for providing a feedback to the user, said feedback depending on the result of the geometric evaluation,

wherein at least one of the class labels identifies an evaluation point, the evaluation point being selected from the group comprising a start point of a motion phase, an end point of a motion phase, and an intermediate point in time of a motion phase where a reversal of a movement occurs, and

wherein the geometric evaluation data comprises geometric evaluation data for performing a geometric evaluation of a configuration of key data elements of a particular image data structure that corresponds to said evaluation point.

16. An evaluation data structure configured for evaluating a specific motion pattern in a sequence of image data structures, the specific motion pattern corresponding to a particular physical exercise and the evaluation data structure comprising:

a machine learning (ML) model artifact of an exercise specific ML model configured for evaluating the particular physical exercise,

wherein the ML model is trained based on a plurality of sequences of image data structures showing different variants of the specific motion pattern for the particular physical exercise,

wherein the ML model is configured to:

determine, based on input data comprising key data elements provided for at least one image data structure, class labels for each image data structure, said class labels identifying at least one of: at least one motion phase of the specific motion pattern and at least one evaluation point of the specific motion pattern, wherein the key data elements indicate positions of landmarks in the image data structures;

generate geometric evaluation data and perform a geometric evaluation of a configuration of key data elements of a particular image data structure that corresponds to an evaluation point or for performing a geometric evaluation of at least one motion phase of the specific motion pattern,

feedback data for providing a feedback to the user, said feedback depending on the result of the geometric evaluation,

wherein at least one of the class labels identifies an evaluation point, the evaluation point being selected from the group comprising a start point of a motion phase, an end point of a motion phase, and an intermediate point in time of a motion phase where a reversal of a movement occurs, and

wherein the geometric evaluation data comprises geometric evaluation data for performing a geometric evaluation of a configuration of key data elements of a particular image data structure that corresponds to said evaluation point.

17. A database comprising at least one evaluation data structure according to claim 16 .

18. A mobile device,

the mobile device being configured for downloading at least one evaluation data structure from a web server, each evaluation data structure being an evaluation data structure according to claim 16 , wherein the evaluation data structure is configured for evaluating a corresponding specific motion pattern;

wherein the mobile device is configured for evaluating a sequence of image data structures using at least one of said evaluation data structures downloaded from the web server.

19. A method for evaluating a motion pattern on a mobile device, the method comprising:

downloading at least one evaluation data structure from a web server, each evaluation data structure being an evaluation data structure according to claim 16 , wherein the evaluation data structure is configured for evaluating a corresponding specific motion pattern;

evaluating a motion pattern in a sequence of image data structures using at least one of the evaluation data structures downloaded from the web server.

Assignments (2)
CHANGE OF ADDRESS Recorded Aug 10, 2022
From: KAIA HEALTH SOFTWARE GMBH
To: KAIA HEALTH SOFTWARE GMBH
Reel/Frame 061143/0734 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2020
From: MEHL, KONSTANTIN; STROBEL, MAXIMILIAN
To: KAIA HEALTH SOFTWARE GMBH
Reel/Frame 051781/0343 →
Continuity (1)
Related Publication 20210209349A1 · Jul 8, 2021
Cited By (1)
US 12,518,397