Systems and methods for markerless tracking of subjects

1. A system for markerless tracking of a subject performing a motor exercise, comprising:

an active 3D infrared camera for capturing depth data of a body of the subject during the motor exercise;

a memory for pre-storing a movement profile including a plurality of benchmarks; and

an electronic processor configured to

extract body motion data for the subject's body from the captured depth data, the body motion data including a set of 3D coordinates for a plurality of body joints for each frame of the captured depth data,

detect movements of the subject's body using the body motion data,

determine a set of attributes for the detected movements of the subject's body using the body motion data,

assign a rating for the motor exercise by comparing the set of attributes for the detected movements of the subject's body with the plurality of benchmarks included in the pre-stored movement profile, and

create a session record for the subject including the body motion data, the set of attributes for the detected movements of the subject's body, and the assigned rating for the motor exercise,

wherein the set of attributes is a first set of attributes, wherein the electronic processor is further configured to

retrieve a second set of attributes from the memory, the second set of attributes having been stored in the memory prior to determining the first set of attributes,

determine a difference between the first set of attributes and the second set of attributes

determine whether the subject is in an ON functional state or in an OFF functional state based on the determined difference between the first set of attributes and the second set of attributes, and

add subject state data to the session record for the subject, the subject state data indicating the determined functional state of the subject.

2. The system of claim 1 , wherein the set of attributes for the detected movements of the subject's body including at least one selected from a group consisting of speed, distance, and amplitude.

3. The system of claim 1 , further comprising an RGB camera for capturing a sequence of color images of the subject performing the motor exercise, wherein the RGB camera is positioned at a predetermined location relative to the location of the active 3D infrared camera, wherein the RGB camera and the active 3D infrared camera have overlapping fields of view of the subject, wherein for each color image in the sequence of color images, the electronic processor is configured to

calculate 2D coordinates in the color image for each of the plurality of body joints using the body motion data and prior knowledge of the locations and the fields of view of the cameras,

alter the color image by superimposing a skeleton on the color image, the skeleton including a visual indicator at the calculated 2D coordinates for each of the plurality of body joints of the subject's body, and

add the altered color image to the session record for the subject.

4. The system of claim 1 , wherein a frame rate of the captured depth data is at least thirty frames per second.

5. The system of claim 1 , wherein the motor exercise including at least one motor exercise of the unified Parkinson's disease rating scale.

6. The system of claim 1 , wherein the rating is a first rating, wherein the system further comprising a user interface, wherein the electronic processor is further configured to

receive a second rating for the motor exercise via the user interface,

determine a difference between the first rating for the motor exercise and the second rating for the motor exercise, and

adjust at least one of the plurality of benchmarks included in the pre-stored movement profile based on the determined difference between the first rating for the motor exercise and the second rating for the motor exercise.

7. The system of claim 1 , wherein the electronic processor is further configured to

extract facial motion data for a face of the subject from the captured depth data, the facial motion data including a set of 3D coordinates for a plurality of facial points for each frame of the captured depth data, and

add the facial motion data to the session record for the subject.

8. The system of claim 1 , wherein the electronic processor is further configured to

extract facial motion data for a face of the subject from the captured depth data, the facial motion data including a set of 3D coordinates for a plurality of facial points for each frame of the captured depth data,

determine identification data for the subject based on the facial motion data, and

add the determined identification data for the subject to the session record for the subject.

9. The system of claim 1 , wherein each of the plurality of benchmarks define a set of conditions associated with a mobility level, wherein the electronic processor assigning the rating for the motor exercise by comparing the set of attributes for the detected movements of the subject's body with the plurality of benchmarks included in the pre-stored movement profile including

selecting a lowest mobility level with at least one condition met by the set of attributes for the detected movements of the subject's body, and

assigning the rating for the motor exercise to the selected mobility level.

10. A system for markerless tracking of subjects located in a space, comprising:

an active 3D infrared camera for capturing depth data in the space; and

an electronic processor configured to

extract body motion data for a body of a first subject located in the space from the captured depth data, the body motion data including a set of 3D coordinates for a plurality of body joints for each frame of the captured depth data,

detect movements of the first subject's body using the body motion data,

determine a set of attributes for the detected movements of the first subject's body using the body motion data,

extract facial motion data for a face of the first subject from the captured depth data, the facial motion data including a set of 3D coordinates for a plurality of facial points for each frame of the captured depth data,

determine identification data for the first subject using the facial motion data, and

create a session record for the first subject including the body motion data, the detected movements of the first subject's body, the set of attributes, and the identification data for the first subject,

wherein the electronic processor is further configured to

extract second body motion data for a second body of a second subject located in the space from the captured depth data, the second body motion data including a set of 3D coordinates for a second plurality of body joints for each frame of the captured depth data,

detect movements of the second subject's body using the second body motion data,

determine a second set of attributes for the detected movements of the second subject's body based on the second body motion data,

extract second facial motion data for a second face of the second subject face from the captured depth data, the second facial motion data including a set of 3D coordinates for a second plurality of facial points for each frame of the captured depth data,

determine identification data for the second subject based on the second facial motion data, and

create a second session record for the second subject including the second body motion data, the detected movements of the second subject's body, the second set of attributes, and the identification data for the second subject.

11. The system of claim 10 , wherein the set of attributes for the detected movements of the first subject's body including at least one selected from a group consisting of speed, distance, and amplitude.

12. The system of claim 10 , further comprising an RGB camera for capturing a sequence of color images of the first subject, wherein the RGB camera is positioned at a predetermined location relative to the location of the active 3D infrared camera, wherein the RGB camera and the active 3D infrared camera have overlapping fields of view of the first subject, wherein for each color image in the sequence of color images, the electronic processor is configured to

calculate 2D coordinates in the color image for each of the plurality of body joints using the body motion data and prior knowledge of the locations and the fields of view of the cameras,

alter the color image by superimposing a skeleton on the color image, the skeleton including a visual indicator at the calculated 2D coordinates for each of the plurality of body joints, and

add the altered color image to the session record for the first subject.

13. The system of claim 10 , wherein a frame rate of the captured depth data is at least thirty frames per second.

14. A method for markerless tracking of a subject performing a motor exercise, comprising:

pre-storing a movement profile in a memory, the pre-stored movement profile including a plurality of benchmarks;

capturing, with an active 3D infrared camera, depth data of a body of the subject during the motor exercise;

extracting, with an electronic processor, body motion data for the subject's body from the captured depth data, the body motion data including a set of 3D coordinates for a plurality of body joints for each frame of the captured depth data;

detecting, with the electronic processor, movements of the subject's body using the body motion data;

determining, with the electronic processor, a set of attributes for the detected movements of the subject's body using the body motion data;

assigning, with the electronic processor, a rating for the motor exercise by comparing the set of attributes for the detected movements of the subject's body with the plurality of benchmarks included in the pre-stored movement profile; and

creating, with the electronic processor, a session record for the subject including the body motion data, the set of attributes for the detected movements of the subject's body, and the assigned rating for the motor exercise,

wherein the set of attributes is a first set of attributes, wherein the method further comprising

retrieving a second set of attributes from the memory, the second set of attributes having been stored prior to determining the first set of attributes;

determining, with the electronic processor, a difference between the first set of attributes and the second set of attributes;

determining, with the electronic processor, whether the subject is in an ON functional state or in an OFF functional state based on the determined difference between the first set of attributes and the second set of attributes; and

adding, with the electronic processor, subject state data to the session record for the subject, the subject state data indicating the determined functional state of the subject.

15. The method of claim 14 , wherein the set of attributes for the detected movements of the subject's body including at least one selected from a group consisting of speed, distance, and amplitude.

16. The method of claim 14 , wherein the motor exercise including at least one motor exercise of the unified Parkinson's disease rating scale.

17. The method of claim 14 , wherein each of the plurality of benchmarks define a set of conditions associated with a mobility level, wherein assigning the rating for the motor exercise by comparing the set of attributes for the detected movements of the subject's body with the plurality of benchmarks included in the pre-stored movement profile including

selecting a lowest mobility level with at least one condition met by the set of attributes for the detected movements of the subject's body, and

assigning the rating for the motor exercise to the selected mobility level.