Granted Patent B2

US 9,503,786 · App. 14/821,896 · Granted Nov 22, 2016

Inventors: Rana el Kaliouby (Milton, MA); Abdelrahman Mahmoud (Somerville, MA); Panu James Turcot (San Francisco, CA)

Assignee: Affectiva, Inc.

H04N21/4668A61B5/165G06Q30/0631H04N21/251H04N21/4223H04N21/44218H04N21/4667G06K2009/00328

Abstract

Claims (40)

1. A computer-implemented method for affect based recommendations comprising:

playing a first media presentation to an individual;

capturing mental state data, wherein the mental state data includes facial data, for the individual, while the first media presentation is played;

comparing the mental state data that was captured against mental state event temporal signatures;

inferring mental states, using one or more processors, based on the mental state data which was collected and analysis of the facial data for at least brow furrows;

correlating the mental state data which was captured for the individual to mental state data collected from other people who experienced the first media presentation,

wherein the correlating is based on identifying and using maximally dissimilar responses during part of the correlating;

ranking the first media presentation relative to another media presentation based on the mental state data which was captured, wherein the ranking is for the individual based on the mental state data captured from the individual; and

recommending a second media presentation to the individual based on the mental state data for the individual which was captured wherein the recommending the second media presentation to the individual is further based on the correlating between the individual and the other people.

2. The method of claim 1 further comprising identifying a mental state event type based on the comparing.

3. The method of claim 2 wherein the recommending of the second media presentation is further based on the mental state event type.

4. The method of claim 1 wherein the first media presentation includes a first socially shared livestream video.

5. The method of claim 4 further comprising generating highlights for the first socially shared livestream video, based on the mental state data that was captured.

6. The method of claim 4 wherein the first socially shared livestream video includes an overlay with information on the mental state data that was captured.

7. The method of claim 6 wherein the overlay includes information on the mental state data collected from the other people.

8. The method of claim 4 wherein the mental state data that was captured for the first socially shared livestream video is analyzed substantially in real time.

9. The method of claim 4 wherein the second media presentation includes a second socially shared livestream video.

10. The method of claim 9 further comprising a recommendation for changing from the first socially shared livestream video to the second socially shared livestream video.

11. The method of claim 9 wherein the first socially shared livestream video is broadcast to a plurality of people.

12. The method of claim 11 further comprising providing an indication to the individual that the second socially shared livestream video is ready to be joined.

13. The method of claim 1 further comprising analyzing the mental state data to produce mental state information.

14. The method according to claim 1 wherein the first media presentation includes one of a group consisting of a movie, a television show, a web series, a webisode, a video, a video clip, an electronic game, an e-book, and an e-magazine.

15. The method according to claim 1 wherein the second media presentation includes one of a group consisting of a movie, a television show, a web series, a webisode, a video, a video clip, an electronic game, an e-book, and an e-magazine.

16. The method according to claim 1 wherein the first media presentation is played on a web-enabled interface.

17. A computer program product embodied in a non-transitory computer readable medium comprising:

code for playing a first media presentation to an individual;

code for capturing mental state data, wherein the mental state data includes facial data, for the individual while the first media presentation is played;

code for inferring mental states, executed on one or more processors, based on the mental state data which was collected and analysis of the facial data for at least brow furrows;

code for correlating the mental state data which was captured for the individual to mental state data collected from other people who experienced the first media presentation, wherein the correlating is based on identifying and using maximally dissimilar responses during part of the correlating;

code for ranking the first media presentation relative to another media presentation based on the mental state data which was captured; and

code for recommending a second media presentation to the individual based on the mental state data for the individual which was captured wherein the recommending the second media presentation to the individual is further based on the correlating between the individual and the other people.

18. A computer system for affect based recommendations comprising:

a memory for storing instructions;

one or more processors attached to the memory wherein the one or more processors are configured to:

play a first media presentation to an individual;

capture mental state data, wherein the mental state data includes facial data, for the individual while the first media presentation is played;

infer mental states, using the one or more processors, based on the mental state data which was collected and analysis of the facial data for at least brow furrows;

correlate the mental state data which was captured for the individual to mental state data collected from other people who experienced the first media presentation, wherein the correlating is based on identifying and using maximally dissimilar responses during part of the correlating;

rank the first media presentation relative to another media presentation based on the mental state data which was captured; and

recommend a second media presentation to the individual based on the mental state data for the individual which was captured wherein recommendation of the second media presentation to the individual is further based on correlation between the individual and the other people.

Assignments (1)

Continuity (29)