Method and apparatus for tracking of food intake and other behaviors and providing relevant feedback
A sensing device monitors and tracks food intake events and details. A processor, appropriately programmed, controls aspects of the sensing device to capture data, store data, analyze data and provide suitable feedback related to food intake. More generally, the methods might include detecting, identifying, analyzing, quantifying, tracking, processing and/or influencing, related to the intake of food, eating habits, eating patterns, and/or triggers for food intake events, eating habits, or eating patterns. Feedback might be targeted for influencing the intake of food, eating habits, or eating patterns, and/or triggers for those. The sensing device can also be used to track and provide feedback beyond food-related behaviors and more generally track behavior events, detect behavior event triggers and behavior event patterns and provide suitable feedback.
1. A computer-based method of detecting performance of physical gestures from data provided by sensors, the method comprising:
obtaining raw sensor data from at least one sensor of the sensors;
determining, from the raw sensor data, a macro signature data structure for a gesture having a gesture time envelope;
generating a gesture envelope dataset for the gesture using at least the macro signature data structure, wherein the gesture envelope dataset comprises a gesture start time and a gesture end time that delimit the gesture time envelope, and a gesture anchor time that correlates to a statistically determined value derived based on the raw sensor data and exceeding a threshold, the gesture anchor time occurring at a single time point between the gesture start time and the gesture end time; and
processing the gesture envelope dataset to identify a gesture label to be associated with the gesture.
2. The method of claim 1 , wherein processing the gesture envelope dataset comprises determining whether the gesture corresponds to a monitored gesture.
3. The method of claim 2 , wherein determining whether the gesture corresponds to the monitored gesture comprises applying the gesture envelope dataset as an input to a trained classifier.
4. The method of claim 1 , further comprising outputting the gesture label, the gesture label being indicative of a detected physical gesture corresponding to the raw sensor data.
5. The method of claim 1 , wherein the statistically determined value comprises a maximum, a minimum, a mean, or a standard deviation of the raw sensor data.
6. The method of claim 1 , wherein the gesture envelope dataset further comprises in-envelope sensor data, wherein in-envelope sensor data comprises portions of the raw sensor data that occurred within the gesture time envelope.
7. The method of claim 6 , further comprising determining, from the raw sensor data and the macro signature data structure, a feature value determined from a feature expression that is a function of the raw sensor data, and determined from at least portions of the raw sensor data obtained between the gesture start time and the gesture end time, wherein the feature value is representative of a defining characteristic for the raw sensor data in the macro signature data structure.
8. The method of claim 7 , wherein generating the gesture envelope dataset comprises ignoring raw sensor data having an associated time that is outside at least a portion of the macro signature data structure, the gesture envelope dataset comprising the feature value.
9. The method of claim 1 , wherein the at least one sensor is part of a wearable device.
10. The method of claim 1 , further comprising identifying an activity or an event that corresponds to the gesture.
11. The method of claim 1 , further comprising selecting the gesture label from a predefined set of gesture labels.
12. The method of claim 1 , wherein:
generating the gesture envelope dataset comprises computing one or more feature values each determined from a corresponding feature expression, one or more time values of the macro signature data structure, and the raw sensor data; and
the one or more feature values comprise at least one of: a total duration associated with the gesture start time and the gesture end time, a time elapsed since a last prior gesture, or a time delay until a next gesture.
13. The method of claim 1 , wherein processing the gesture envelope dataset comprises using one or more of a trained classifier, an unsupervised classification process, or a supervised classification process.
14. One or more processor-readable media storing instructions which, when executed by one or more processors, cause performance of:
obtaining sensor data from at least one sensor;
determining timing information of a gesture, wherein the timing information comprises an action start time and an action end time;
generating gesture data based on at least a subset of the sensor data corresponding to the action start time and the action end time;
detecting a physical gesture corresponding to the sensor data based on processing the gesture data using a statistically determined value derived from the at least the subset of the sensor data which occurs at a single time point between the action start time and the action end time and exceeds a threshold; and
identifying a gesture label to be associated with the detected physical gesture based on the gesture occurring within the timing information and corresponding to a particular gesture.
15. The one or more processor-readable media of claim 14 , wherein processing the gesture data comprises determining whether the gesture corresponds to a monitored gesture by applying the gesture data as an input to a trained classifier.
16. The one or more processor-readable media of claim 14 , wherein the instructions, when executed by the one or more processors, cause performance of determining, from the sensor data and the timing information, a feature value determined from a feature expression that is a function of the sensor data, and determined from at least portions of the sensor data obtained between the action start time and the action end time, wherein the feature value represents a defining characteristic for the sensor data in the timing information; and
wherein generating the gesture data comprises ignoring all of the sensor data having an associated time that is outside at least a portion of the timing information, the gesture data comprising the feature value.
17. The one or more processor-readable media of claim 14 , wherein the instructions, when executed by the one or more processors, cause performance of outputting the gesture label, the gesture label being indicative of the detected physical gesture corresponding to the sensor data.
18. The one or more processor-readable media of claim 14 , further comprising:
grouping a plurality of gestures, including the gesture, to an event data structure corresponding to an activity or event; and
identifying an activity or an event that corresponds to the event data structure, distinct from an activity or an event that would correspond to the gesture alone.
19. The one or more processor-readable media of claim 18 , wherein grouping the plurality of gestures comprises:
maintaining confidence levels and gesture labels for gestures of the plurality of gestures, the confidence levels including a confidence level for each gesture of the plurality of gestures and the gesture labels including a gesture label for each gesture of the plurality of gestures; and
modifying the confidence levels of the gestures of the plurality of gestures based on temporal spacing and sequence of predicted activities represented by the plurality of gestures.
20. A system configured to identify physical gestures, the system comprising:
one or more sensors;
one or more processors; and
one or more processor-readable media storing instructions which, when executed by the one or more processors, cause performance of one or more operations including:
obtaining sensor data derived from the one or more sensors;
determining timing information of a gesture, wherein the timing information comprises an action start time and an action end time;
generating a gesture data based on at least a subset of the sensor data corresponding to the action start time and the action end time;
detecting a physical gesture corresponding to the sensor data based on processing the gesture data using a statistically determined value derived based on the at least the subset of the sensor data which occurs at a single time point between the action start time and the action end time and exceeds a threshold; and
identifying a gesture label to be associated with the detected physical gesture based on the gesture occurring within the timing information and corresponding to a particular gesture.