IP Library › Granted Patent US 10,799,168
Granted Patent B2
US 10,799,168 · App. 15/720,301 · Granted Oct 13, 2020

Individual data sharing across a social network

Inventors: Rana el Kaliouby (Milton, MA); Richard Scott Sadowsky (Sturbridge, MA); Oliver Orion Wilder-Smith (Holliston, MA)
Assignee: Affectiva, Inc.
A61B5/165G06K9/6201G06Q10/101G06Q50/01G16H40/63G16H40/67G16H50/20H04L67/306
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,799,168
App. No.
15/720,301
Granted
Oct 13, 2020
Kind
B2
Abstract

Facial image data of an individual is collected of the individual to provide mental state data using a first web-enabled computing device. The mental state data is analyzed to produce mental state information using a second web-enabled computing device. The mental state information is shared across a social network using a third web-enabled computing device. The mental state data is also collected from the individual through capture of sensor information. The mental state data is also collected from the individual through capture of audio data. The individual elects to share the mental state information across the social network. The mental state data may be collected over a period of time and analyzed to determine a mood of the individual. The mental state information is translated into a representative icon for sharing, which may include an emoji. An image of the individual is shared along with the mental state information.

Claims (52)

1. A computer-implemented method for data sharing comprising:

collecting mental state data of an individual, wherein the mental state data includes facial image data of the individual;

analyzing the mental state data using a neural network to produce mental state information and a plurality of emotional intensity metrics, wherein the neural network was trained to identify facial expressions and mental states;

sharing the mental state information across a social networking platform;

coalescing the plurality of emotional intensity metrics into a summary intensity metric;

inferring mental states based on both the mental state data which was collected and the summary intensity metric;

identifying a user on the social networking platform who is associated with at least one mental state similar to at least one of the inferred mental states, wherein the identifying a user on the social networking platform includes searching a group specifically created to share mental state information; and

providing an identification of the user to the individual as having a similar mental state to at least one of the inferred mental states.

2. The method of claim 1 further comprising electing, by the individual, to share the mental state information.

3. The method according to claim 2 further comprising presenting the mental state information to the individual, prior to the electing.

4. The method of claim 1 wherein the mental state data is collected over a period of time and the mental state information that is shared is a reflection of a mood for the individual.

5. The method of claim 4 wherein the mood includes one of a group comprising frustration, confusion, disappointment, hesitation, cognitive overload, focusing, being engaged, attending, boredom, exploration, confidence, trust, delight, and satisfaction.

6. The method of claim 1 wherein the sharing includes posting mental state information to a social networking platform web page.

7. The method of claim 1 further comprising uploading the mental state information to a server.

8. The method of claim 1 further comprising distributing the mental state information across a computer network.

9. The method of claim 1 wherein the mental state data includes one of a group comprising physiological data, facial data, or actigraphy data.

10. The method of claim 9 wherein a webcam is used to capture one or more of the facial data or the physiological data.

11. The method of claim 9 wherein the physiological data includes one or more of electrodermal activity, heart rate, heart rate variability, skin temperature, and respiration.

12. The method according to claim 1 further comprising communicating an image of the individual with the mental state information that is being shared.

13. The method according to claim 12 wherein the image of the individual is from a peak time of mental state activity.

14. The method of claim 1 further comprising restricting distribution of the mental state information to a subset of the social networking platform.

15. The method of claim 1 further comprising sharing aggregated mental state information across the social networking platform.

16. The method according to claim 1 wherein the mental state data is collected as the individual interacts with a web-enabled application.

17. The method according to claim 16 further comprising forwarding a reference to the web-enabled application as a part of the sharing of the mental state information.

18. The method of claim 17 wherein the forwarding includes a video of material from the web-enabled application.

19. The method according to claim 16 wherein the sharing is part of a rating system for the web-enabled application.

20. The method of claim 1 further comprising translating the mental state information into a representative icon for sharing across the social networking platform.

21. The method of claim 1 wherein the collecting mental state data of an individual includes collecting audio data.

22. The method of claim 21 wherein the analyzing the mental state data of the individual includes analyzing the audio data that was collected.

23. A computer program product embodied in a non-transitory computer readable medium for data sharing, the computer program product comprising code which causes one or more processors to perform operations of:

collecting mental state data of an individual, wherein the mental state data includes facial image data of the individual;

analyzing the mental state data of the individual using a neural network to produce mental state information and a plurality of emotional intensity metrics, wherein the neural network was trained to identify facial expressions and mental states;

sharing the mental state information across a social networking platform;

coalescing the plurality of emotional intensity metrics into a summary intensity metric;

inferring mental states based on both the mental state data which was collected and the summary intensity metric;

identifying a user on the social networking platform who is associated with at least one mental state similar to at least one of the inferred mental states, wherein the identifying a user on the social networking platform includes searching a group specifically created to share mental state information; and

providing an identification of the user to the individual as having a similar mental state to at least one of the inferred mental states.

24. A system for data sharing:

a memory for storing instructions;

one or more processors attached to the memory wherein the one or more processors are configured to:

collect mental state data of an individual, wherein the mental state data includes facial image data of the individual;

analyze the mental state data of the individual using a neural network to produce mental state information and a plurality of emotional intensity metrics, wherein the neural network was trained to identify facial expressions and mental states;

share the mental state information across a social networking platform;

coalesce the plurality of emotional intensity metrics into a summary intensity metric;

infer mental states based on both the mental state data which was collected and the summary intensity metric;

identify a user on the social networking platform who is associated with at least one mental state similar to at least one of the inferred mental states, wherein identifying a user on the social networking platform includes searching a group specifically created to share mental state information; and

provide an identification of the user to the individual as having a similar mental state to at least one of the inferred mental states.

25. The method of claim 1 wherein the neural network comprises a multilayered analysis engine used for emotional analysis.

26. The method of claim 1 wherein the neural network is a convolutional neural network.

27. The method of claim 1 wherein the neural network provides an output to a pooling layer.

28. The method of claim 1 wherein the identifying a user on the social networking platform includes searching a contact list for the individual.

29. The method of claim 28 wherein the identifying a user on the social networking platform includes searching an extended contact list for the individual, wherein the extended contact list includes contact lists for users in the contact list for the individual.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 16, 2018
From: EL KALIOUBY, RANA; SADOWSKY, RICHARD SCOTT; WILDER-SMITH, OLIVER ORION
To: AFFECTIVA, INC.
Reel/Frame 045251/0283 →
Continuity (49)
Continuation In Part 13297342 · Nov 16, 2011
Continuation In Part 15395750 · Dec 30, 2016
Continuation In Part 15262197 · Sep 12, 2016
Continuation In Part 14796419 · Jul 10, 2015
Continuation In Part 13153745 · Jun 6, 2011
Continuation In Part 14460915 · Aug 15, 2014
Continuation In Part 13153745 · Jun 6, 2011
Provisional Application 62439928 · Dec 29, 2016
Provisional Application 62442325 · Jan 4, 2017
Provisional Application 62448448 · Jan 20, 2017
Provisional Application 62442291 · Jan 4, 2017
Provisional Application 62469591 · Mar 10, 2017
Provisional Application 62503485 · May 9, 2017
Provisional Application 62524606 · Jun 25, 2017
Provisional Application 61414451 · Nov 17, 2010
Provisional Application 61439913 · Feb 6, 2011
Provisional Application 61447089 · Feb 27, 2011
Provisional Application 61447464 · Feb 28, 2011
Provisional Application 61467209 · Mar 24, 2011
Provisional Application 61549560 · Oct 20, 2011
Provisional Application 62273896 · Dec 31, 2015
Provisional Application 62301558 · Feb 29, 2016
Provisional Application 62370421 · Aug 3, 2016
Provisional Application 62217872 · Sep 12, 2015
Provisional Application 62222518 · Sep 23, 2015
Provisional Application 62265937 · Dec 10, 2015
Provisional Application 62273896 · Dec 31, 2015
Provisional Application 62301558 · Feb 29, 2016
Provisional Application 62370421 · Aug 3, 2016
Provisional Application 62023800 · Jul 11, 2014
Provisional Application 62047508 · Sep 8, 2014
Provisional Application 62082579 · Nov 20, 2014
Provisional Application 62128974 · Mar 5, 2015
Provisional Application 61352166 · Jun 7, 2010
Provisional Application 61388002 · Sep 30, 2010
Provisional Application 61414451 · Nov 17, 2010
Provisional Application 61439913 · Feb 6, 2011
Provisional Application 61867007 · Aug 16, 2013
Provisional Application 61924252 · Jan 7, 2014
Provisional Application 61916190 · Dec 14, 2013
Provisional Application 61927481 · Jan 15, 2014
Provisional Application 61953878 · Mar 16, 2014
Provisional Application 61972314 · Mar 30, 2014
Provisional Application 62023800 · Jul 11, 2014
Provisional Application 61352166 · Jun 7, 2010
Provisional Application 61388002 · Sep 30, 2010
Provisional Application 61414451 · Nov 17, 2010
Provisional Application 61439913 · Feb 6, 2011
Related Publication 20180035938A1 · Feb 8, 2018
Cited By (1)
US 12,332,855