Systems and methods for estimation of Parkinson's Disease gait impairment severity from videos using MDS-UPDRS

1. A method for evaluating motion from a video, the method comprising:

identifying a target individual in a set of one or more frames in a video;

analyzing the set of frames to determine a set of pose parameters;

generating a 3D body mesh based on the pose parameters;

identifying joint positions for the target individual in the set of frames based on the generated 3D body mesh;

predicting a motion evaluation score based on the identified joint positions, wherein predicting the motion evaluation score is performed using a score evaluation model that is trained using a hybrid ordinal-focal objective; and

providing an output based on the motion evaluation score.

2. The method of claim 1 , wherein identifying a target individual comprises:

tracking a plurality of individuals in each frame of the set of frames; and

identifying an individual that appears most often in the set of frames as the target individual.

3. The method of claim 1 , wherein identifying a target individual comprises identifying a bounding box for the target individual in each of the set of frames, wherein analyzing the set of frames comprises analyzing the bounding box in each of the set of frames.

4. The method of claim 1 , wherein the score evaluation model takes as input at least one of a joint collection distance (JCD) and two-scale motion features.

5. The method of claim 4 , wherein the JCD and two-scale motion features are embedded into latent vectors at each frame through a series of convolutions to learn joint correlation and reduce an effect of skeleton noise.

6. The method of claim 1 , wherein the motion evaluation score is a Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS) gait score.

7. The method of claim 1 , further comprising using a trained temporal convolutional neural network (TCNN) on a sequence of 3D poses to quantify movement-linked disease markers.

8. The method of claim 7 , wherein the TCNN uses a Rater Confusion Estimation (RCE) framework that jointly learns a rater scoring noise by creating a learnable confusion matrix (CM) for each rater, and wherein the method further comprises optimizing, the CM while classifying input videos using an ordinal focal strategy.

9. The method of claim 1 , further comprising extracting a 2D hand skeleton of the target individual to produce a number of keypoints for the hand and using the keypoints to quantify motor impairments.

10. A system for evaluating motion from a video, the system comprising:

at least one processor; and

memory containing a motion evaluation application, wherein the motion evaluation application configures the at least one processor to:

identify a target individual in a set of one or more frames in a video;

analyze the set of frames to determine a set of pose parameters;

generate a 3D body mesh based on the pose parameters;

identify joint positions for the target individual in the set of frames based on the generated 3D body mesh;

predict a motion evaluation score based on the identified joint positions, wherein predicting the motion evaluation score is performed using a score evaluation model that is trained using a hybrid ordinal-focal objective; and

provide an output based on the motion evaluation score.

11. The system of claim 10 , wherein identifying a target individual comprises:

tracking a plurality of individuals in each frame of the set of frames; and

identifying an individual that appears most often in the set of frames as the target individual.

12. The system of claim 10 , wherein identifying a target individual comprises identifying a bounding box for the target individual in each of the set of frames, wherein analyzing the set of frames comprises analyzing the bounding box in each of the set of frames.

13. The system of claim 10 , wherein the score evaluation model takes as input at least one of a joint collection distance (JCD) and two-scale motion features.

14. The system of claim 13 , wherein the JCD and two-scale motion features are embedded into latent vectors at each frame through a series of convolutions to learn joint correlation and reduce an effect of skeleton noise.

15. The system of claim 10 , wherein the motion evaluation score is a Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS) gait score.

16. The system of claim 10 , wherein the motion evaluation application configures the at least one processor to use a trained temporal convolutional neural network (TCNN) on a sequence of 3D poses to quantify movement-linked disease markers.

17. The system of claim 16 , wherein the TCNN uses a Rater Confusion Estimation (RCE) framework that jointly learns a rater scoring noise by creating a learnable confusion matrix (CM) for each rater, and wherein the the motion evaluation application further configures the at least one processor to optimize the CM while classifying input videos using an ordinal focal strategy.

18. The system of claim 10 , wherein the motion evaluation application configures the at least one processor to extract a 2D hand skeleton of the target individual to produce a number of keypoints for the hand and using the keypoints to quantify motor impairments.

19. A method for evaluating motion from a video, the method comprising:

identifying a target individual in a set of one or more frames in a video;

analyzing the set of frames to determine a set of pose parameters;

generating a 3D body mesh based on the pose parameters;

identifying joint positions for the target individual in the set of frames based on the generated 3D body mesh;

using a trained temporal convolutional neural network (TCNN) on a sequence of 3D poses to quantify movement-linked disease markers;

predicting a motion evaluation score based on the identified joint positions; and

providing an output based on the motion evaluation score, wherein the TCNN uses a Rater Confusion Estimation (RCE) framework that jointly learns a rater scoring noise by creating a learnable confusion matrix (CM) for each rater; and

optimizing the CM while classifying input videos using an ordinal focal strategy.

20. The method of claim 19 , further comprising extracting a 2D hand skeleton of the target individual to produce a number of keypoints for the hand and using the keypoints to quantify motor impairments.