Sporadic collection with mobile affect data
An individual can exhibit one or more mental states when reacting to a stimulus. A camera or other monitoring device can be used to collect, on an intermittent basis, mental state data including facial data. The mental state data can be interpolated between the intermittent collecting. The facial data can be obtained from a series of images of the individual where the images are captured intermittently. A second face can be identified, and the first face and the second face can be tracked.
1. A computer-implemented method for mental state analysis comprising:
collecting mental state data of an individual on an intermittent basis wherein the mental state data includes facial image data and the facial image data is collected intermittently across a plurality of devices;
imputing additional mental state data where the mental state data is missing, wherein the imputing is based on mental state data collected from other individuals associated with the individual;
interpolating, using one or more processors, mental state data in between the collecting which is intermittent;
obtaining analysis of the mental state data on the individual, wherein the analysis of the mental state data is received from a web service and includes analyzing the facial image data; and
rendering an output based on the analysis of the mental state data.
2. The method of claim 1 wherein the collecting mental state data is based on image analysis of the facial image data.
3. The method of claim 1 wherein the facial image data is obtained from a series of images of the individual.
4. The method of claim 3 further comprising identifying a second face from a second individual within the series of images.
5. The method of claim 4 further comprising tracking a face for the individual within the series of images.
6. The method of claim 5 further comprising tracking the second face within the series of images.
7. The method of claim 1 further comprising collecting other mental state data from the individual on a continuous basis.
8. The method of claim 7 wherein the other mental state data includes electrodermal activity data.
9. The method of claim 7 wherein the other mental state data includes audio voice data.
10. The method of claim 1 wherein the mental state data of an individual on an intermittent basis includes audio voice data.
11. The method of claim 1 wherein the obtaining analysis of the mental state data is based on sending a request to a web service for the analysis.
12. The method of claim 1 further comprising interpolating mental state analysis in between the collecting which is intermittent.
13. The method of claim 1 wherein the facial image data is collected intermittently when the individual is looking in a direction of a camera.
14. The method of claim 13 further comprising performing face detection to determine when the individual is looking in the direction of the camera.
15. The method of claim 14 wherein the face detection is based on image classifiers.
16. The method of claim 1 further comprising comparing the mental state data against a plurality of mental state event temporal signatures.
17. The method of claim 1 further comprising filtering out faces of one or more other people to determine when the individual is looking in a direction of a camera.
18. The method of claim 1 further comprising determining contextual information.
19. The method of claim 18 wherein the contextual information is based on one or more of skin temperature or accelerometer data.
20. The method of claim 18 wherein the contextual information is based on one or more of a photograph, an email, a text message, a phone log, or GPS information.
21. The method of claim 1 further comprising translating the mental state data into an emoji for representation of the individual.
22. The method of claim 1 further comprising interpolating mental state analysis in between the collecting which is intermittent; collecting other mental state data, including electrodermal activity data, from the individual on a continuous basis; imputing additional mental state data where the mental state data is missing; filtering out faces of one or more other people to determine when an individual is looking in a direction of a camera; determining contextual information based on accelerometer data; and sending a request to a web service for the analysis where the analysis of the mental state data is received from a web service.
23. The method of claim 1 further comprising sending one or more of the mental state data, a subset of the mental state data, or an initial analysis of the mental state data to the web service.
24. The method of claim 23 wherein the sending is accomplished on a periodic basis.
25. The method of claim 1 further comprising inferring mental states based on the mental state data which was collected.
26. A computer program product embodied in a non-transitory computer readable medium for mental state analysis, the computer program product comprising code which causes one or more processors to perform operations of:
collecting mental state data of an individual on an intermittent basis wherein the mental state data includes facial image data and the facial image data is collected intermittently across a plurality of devices;
imputing additional mental state data where the mental state data is missing, wherein the imputing is based on mental state data collected from other individuals associated with the individual;
interpolating mental state data in between the collecting which is intermittent;
obtaining analysis of the mental state data on the individual, wherein the analysis of the mental state data is received from a web service and includes analyzing the facial image data; and
rendering an output based on the analysis of the mental state data.
27. A system for mental state analysis comprising:
a memory for storing instructions;
one or more processors attached to the memory wherein the one or more processors are configured to:
collect mental state data of an individual on an intermittent basis wherein the mental state data includes facial image data and the facial image data is collected intermittently across a plurality of devices;
impute additional mental state data where the mental state data is missing, wherein the additional mental state data is based on mental state data collected from other individuals associated with the individual;
interpolate mental state data in between the collecting which is intermittent;
obtain analysis of the mental state data on the individual, wherein the analysis of the mental state data is received from a web service and includes analyzing the facial image data; and
render an output based on the analysis of the mental state data.
28. The method of claim 1 wherein the other individuals are geographically nearby the individual.
29. The method of claim 1 wherein the other individuals are connected with the individual on a computerized social network.