Systems and methods to identify persons and/or identify and quantify pain, fatigue, mood, and intent with protection of privacy
The disclosed technology enables, among other things, the identification of persons and the characterization of mental perceptions (e.g., pain, fatigue, mood) and/or intent (e.g., to perform an action) for medical, safety, home care, and other purposes. Of significance are applications that require long-term patient monitoring, such as tracking disease progression (e.g., multiple sclerosis), or monitoring treatment or rehabilitation efficacy. Therefore, longitudinal data must be acquired over time for the person's identity and other characteristics (e.g., pain level, usage of a cane). However, conventional methods of person identification (e.g., photography) acquire unnecessary personal information, resulting in privacy concerns. The disclosed technology allows measurements to be performed while protecting privacy and functions with partial or incomplete measurements, making it robust to real-world (noisy, uncontrolled) settings, such as in a person's home (whether living alone or with others).
1. A method of identifying and/or quantifying at least one of pain, fatigue, mood, and intent of a person via physical measurements of the person, the method comprising:
receiving, by a processor of a computing device, a data set comprising sensor data acquired by a sensor;
characterizing, by the processor, at least one member selected from the group consisting of movement, activity, and behavior of the person based on the data set, wherein said characterizing comprises:
segmenting one or more persons represented within the data set, from each other and from environmental objects represented within the data set,
calculating at least one feature of at least one of the one or more segmented persons, and
determining a conclusion based on correlating the at least one feature with a relationship;
outputting, by the processor, the conclusion; and
suspending calculating the at least one feature based on at least one of availability of specific features and relevance for the output conclusion.
2. The method of claim 1 , wherein segmenting the one or more persons represented within the data set comprises performing a machine learning and/or pattern recognition technique.
3. The method of claim 1 , wherein the relationship is a learned relationship.
4. The method of claim 3 , wherein the learned relationship is based on a ground truth.
5. The method of claim 1 , wherein the at least one feature comprises physical measurements that are direct measurements.
6. The method of claim 1 , wherein the at least one feature comprises physical measurements that are indirect measurements.
7. The method of claim 1 , wherein the conclusion is based on incomplete measurements.
8. The method of claim 1 , wherein the at least one feature is based on measurements in the data set that are at least one of non-contiguous, non-adjacent, incomplete, inconsistent, non-sequential.
9. The method of claim 1 , wherein the conclusion is based on the consolidation of several different types and combinations of features.
10. The method of claim 1 , wherein the sensor data in the data set are acquired directly from a sensor and/or have undergone additional processing.
11. The method of claim 1 , wherein the data set comprises one or more frames.
12. The method of claim 11 , wherein each frame of the one or more frames corresponds to a snapshot of one or more data streams, comprising data acquired either preceding, or at, a particular moment in time.
13. The method of claim 11 , wherein the one or more frames are captured by a single sensor at multiple points in time and/or captured by multiple sensors.
14. The method of claim 1 , wherein the data set is acquired in substantially real-time, is acquired in batched mode, or is acquired from data previously stored in a database.
15. The method of claim 1 , comprising pre-processing, by the processor, the data set for subsequent analysis, wherein pre-processing comprises one or more steps selected from the group consisting of filtering, conditioning, cleaning, and normalizing the data set.
16. The method of claim 1 , comprising pre-processing, by the processor, the data set for subsequent analysis, wherein pre-processing is performed on raw data from the data set or calculated features.
17. The method of claim 1 , wherein segmenting is performed without any information as to the identity of the segmented person.
18. The method of claim 1 , comprising, after segmenting, determining a label for each of the one or more persons segmented in order to associate calculated features with a specific individual.
19. The method of claim 1 , wherein the at least one feature comprises at least one member selected from the group consisting of: limb length, foot size, head shape, height, body proportion, body volume, voice frequency spectrum, voice volume, stride length, location within a field-of-view, and speed of arm raise.
20. The method of claim 1 , comprising automatically calculating, by the processor, the at least one feature using a machine learning and/or pattern recognition technique.
21. The method of claim 1 , wherein determining, by the processor, the desired characteristic of movement, activity, or behavior based on the data set comprises, before segmenting, pre-processing, by the processor, the data set for subsequent analysis.
22. The method of claim 1 , wherein the relationship is pre-determined.
23. A system for identifying and/or quantifying at least one of pain, fatigue, mood, and intent of a person via physical measurements of the person, the system comprising:
an infrared time-of-flight sensor for acquiring sensor data comprising data corresponding to the person;
a processor; and
a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to:
receive a data set comprising the sensor data acquired by the sensor;
characterize at least one of movement, activity, and behavior of the person based on the data set, by:
segmenting one or more persons represented within the data set, from each other and from environmental objects represented within the data set,
calculating at least one feature of at least one of the one or more segmented persons, and
determining a conclusion of the person based on correlating the at least one feature with a relationship; and
output the conclusion.
24. The system of claim 23 , wherein segmenting the one or more persons represented within the data set comprises performing a machine learning and/or pattern recognition technique.
25. The system of claim 23 , further comprising a display and a housing for the processor and memory.
26. A method of identifying and/or quantifying at least one of pain, fatigue, mood, and intent of a person via physical measurements of the person, the method comprising:
receiving, by a processor of a computing device, a data set comprising sensor data acquired by a sensor;
discarding types of data determined to be non-private;
characterizing, by the processor, at least one member selected from the group consisting of movement, activity, and behavior of the person based on the data set, wherein said characterizing comprises:
segmenting one or more persons represented within the data set, from each other and from environmental objects represented within the data set,
calculating at least one feature of at least one of the one or more segmented persons, and
determining a conclusion based on correlating the at least one feature with a relationship; and
outputting, by the processor, the conclusion.