Granted Patent B2

US 9,204,836 · App. 14/064,136 · Granted Dec 8, 2015

Inventors: Daniel Bender (Cambridge, MA); Rana el Kaliouby (Boston, MA); Evan Kodra (Waltham, MA); Oliver Ernst Nowak (Medford, MA); Richard Scott Sadowsky (Sturbridge, MA)

Assignee: Affectiva, Inc.

A61B5/165A61B5/0022G06F19/3418G06F19/3481A61B5/0077A61B5/02055A61B5/02405A61B5/0531A61B5/0533A61B5/08A61B5/11A61B5/6898A61B5/72G06Q30/0271

Abstract

Claims (48)

1. A computer-implemented method for mental state analysis comprising:

collecting mental state data of an individual on an intermittent basis;

collecting other mental state data, including electrodermal activity data, from the individual on a continuous basis;

obtaining analysis of the mental state data on the individual, as a result of sending a request to a web service for the analysis where the analysis of the mental state data is received from the web service;

interpolating, using one or more processors, mental state analysis in between the collecting which is intermittent;

imputing additional mental state data where the mental state data is missing;

filtering out faces of one or more other people to determine when an individual is looking in a direction of a camera;

determining contextual information based on accelerometer data; and

rendering an output based on the analysis of the mental state data.

2. The method of claim 1 further comprising interpolating mental state data in between the collecting which is intermittent.

3. The method of claim 1 further comprising collecting other mental state data from the individual on a continuous basis.

4. The method of claim 1 wherein the collecting of the mental state data is accomplished with a mobile device.

5. The method of claim 1 wherein the collecting of the mental state data is accomplished for one type of mental state data.

6. The method of claim 5 wherein the one type of mental state data includes facial data.

7. The method of claim 6 wherein the facial data is collected with a webcam.

8. The method of claim 6 wherein the facial data is collected intermittently when the individual is looking in a direction of a camera.

9. The method of claim 8 further comprising performing face detection to determine when the individual is looking in the direction of the camera.

10. The method of claim 8 further comprising filtering out faces of one or more other people to determine when the individual is looking in the direction of the camera.

11. The method of claim 1 further comprising determining contextual information.

12. The method of claim 1 further comprising sending one or more of the mental state data, a subset of the mental state data, or an initial analysis of the mental state data to the web service.

13. The method of claim 1 wherein the mental state data includes one or more of smiles, laughter, smirks, or grimaces.

14. The method of claim 1 further comprising posting the analysis to a social network.

15. A computer program product embodied in a non-transitory computer readable medium for mental state analysis, the computer program product comprising:

code for collecting mental state data of an individual on an intermittent basis;

code for collecting other mental state data, including electrodermal activity data, from the individual on a continuous basis;

code for obtaining analysis of the mental state data on the individual, as a result of sending a request to a web service for the analysis where the analysis of the mental state data is received from the web service;

code for interpolating, using one or more processors, mental state analysis in between the collecting which is intermittent;

code for imputing additional mental state data where the mental state data is missing;

code for filtering out faces of one or more other people to determine when an individual is looking in a direction of a camera;

code for determining contextual information based on accelerometer data; and

code for rendering an output based on the analysis of the mental state data.

16. A system for mental state analysis comprising:

a memory for storing instructions;

one or more processors attached to the memory wherein the one or more processors are configured to:

collect mental state data of an individual on an intermittent basis;

collect other mental state data, including electrodermal activity data, from the individual on a continuous basis;

obtain analysis of the mental state data on the individual, as a result of sending a request to a web service for the analysis where the analysis of the mental state data is received from the web service;

interpolate, using one or more processors, mental state analysis in between the collecting which is intermittent;

impute additional mental state data where the mental state data is missing;

filter out faces of one or more other people to determine when an individual is looking in a direction of a camera;

determine contextual information based on accelerometer data; and

render an output based on the analysis of the mental state data.

17. The computer program product of claim 15 further comprising code for interpolating mental state data in between the collecting which is intermittent.

18. The computer program product of claim 15 further comprising code for collecting other mental state data from the individual on a continuous basis.

19. The computer program product of claim 15 wherein the collecting of the mental state data accomplished for one type of mental state data.

20. The computer program product of claim 19 wherein the one type of mental state data includes facial data.

21. The computer program product of claim 15 wherein the mental state data includes one or more of smiles, laughter, smirks, or grimaces.

22. The computer program product of claim 15 further comprising code for posting the analysis to a social network.

Assignments (1)

Continuity (17)