IP Library Granted Patent US 10,401,860
Granted Patent B2
US 10,401,860 · App. 15/918,122 · Granted Sep 3, 2019

Image analysis for two-sided data hub

Inventors: Jason Krupat (Needham, MA); Rana el Kaliouby (Milton, MA); Jason Radice (Sharon, NH); Gabriele Zijderveld (Somerville, MA); Chilton Lyons Cabot (Dedham, MA)
Assignee: Affectiva, Inc.
G05D1/0088A61B5/0013A61B5/0077A61B5/1176A61B5/165A61B5/18A61B5/4803A61B5/6893G05D1/0221G05D1/0291G06K9/00302G06K9/00845G06K9/4652G06K9/6255G06K9/6269G08G1/0962G10L15/1815G10L15/22G10L25/63A61B2576/02G10L15/1807G10L25/90
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,401,860
App. No.
15/918,122
Granted
Sep 3, 2019
Kind
B2
Abstract

Image analysis is performed for a two-sided data hub. Data reception on a first computing device is enabled by an individual and a content provider. Cognitive state data including facial data on the individual is collected on a second computing device. The cognitive state data is analyzed on a third computing device and the analysis is provided to the individual. The cognitive state data is evaluated and the evaluation is provided to the content provider. A mood dashboard is displayed to the individual based on the analyzing. The individual opts in to enable data reception for the individual. The content provider provides content via a website.

Claims (39)

1. A computer-implemented method for image analysis comprising:

collecting cognitive state data including facial data on an individual in a first vehicle;

analyzing the cognitive state data using a neural network trained to identify facial expressions and cognitive states;

generating a cognitive state profile for the individual in the first vehicle based on the analyzing;

transmitting the cognitive state profile to a two-sided data hub; and

providing vehicular manipulation of a second vehicle from the two-sided data hub.

2. The method of claim 1 wherein the collecting cognitive state data, including facial data on the individual, is accomplished using a webcam.

3. The method of claim 2 wherein the collecting cognitive state data further includes capturing physiological data.

4. The method of claim 1 wherein the collecting of cognitive state data is accomplished within the first vehicle.

5. The method of claim 1 wherein the analyzing further comprises comparing the cognitive state data with collected cognitive state data from other individuals.

6. The method of claim 5 wherein the comparing provides quantified self-information on cognitive states for the individual.

7. The method of claim 1 wherein the analyzing includes using an image classifier from a plurality of image classifiers to detect emotional content within the facial data for the individual.

8. The method of claim 1 wherein the analyzing the cognitive state data is used to enable the individual to track cognitive health.

9. The method of claim 1 further comprising augmenting the analyzing using audio data from the individual.

10. The method of claim 9 wherein the audio data includes voice data from the individual.

11. The method of claim 1 wherein the cognitive state data is indicative of drowsiness, fatigue, distraction, impairment, sadness, stress, happiness, anger, frustration, confusion, disappointment, hesitation, cognitive overload, focusing, engagement, attention, boredom, exploration, confidence, trust, delight, disgust, skepticism, doubt, satisfaction, excitement, laughter, calmness, curiosity, humor, depression, envy, sympathy, embarrassment, poignancy, or mirth.

12. A computer program product embodied in a non-transitory computer readable medium for image analysis, the computer program product comprising code which causes one or more processors to perform operations of:

collecting cognitive state data including facial data on an individual in a first vehicle;

analyzing the cognitive state data using a neural network trained to identify facial expressions and cognitive states;

generating a cognitive state profile for the individual in the first vehicle based on the analyzing;

transmitting the cognitive state profile to a two-sided data hub; and

providing vehicular manipulation of a second vehicle from the two-sided data hub.

13. A computer system for image analysis comprising:

a memory which stores instructions;

one or more processors attached to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to:

collect cognitive state data including facial data on an individual in a first vehicle;

analyze the cognitive state data using a neural network trained to identify facial expressions and cognitive states;

generate a cognitive state profile for the individual in the first vehicle based on the analyzing;

transmit the cognitive state profile to a two-sided data hub; and

provide vehicular manipulation of a second vehicle from the two-sided data hub.

14. The method of claim 1 , further comprising providing driving directions to the individual; identifying a change in a cognitive state of the individual; modifying the driving directions as a function of the change; and providing the modified driving directions to the individual.

15. The method of claim 1 , further comprising:

collecting further cognitive state data including facial data on an individual in the second vehicle;

analyzing the further cognitive state data using a neural network trained to identify facial expressions and cognitive states;

generating a cognitive state profile for the individual in the second vehicle based on the analyzing the further cognitive state data;

transmitting the cognitive state profile for the individual in the second vehicle to the two-sided data hub; and

providing vehicular manipulation of the first or second vehicle from the two-sided data hub.

16. The method of claim 1 wherein the vehicular manipulation includes adjusting an interior temperature.

17. The method of claim 1 wherein the vehicular manipulation includes brake activation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 12, 2018
From: KRUPAT, JASON; EL KALIOUBY, RANA; RADICE, JASON; ZIJDERVELD, GABRIELE; CABOT, CHILTON LYONS
To: AFFECTIVA, INC.
Reel/Frame 045175/0171 →
Continuity (51)
Continuation In Part 15886275 · Feb 1, 2018
Continuation 15875644 · Jan 19, 2018
Continuation In Part 15273765 · Sep 23, 2016
Continuation In Part 14796419 · Jul 10, 2015
Continuation In Part 14460915 · Aug 15, 2014
Continuation In Part 13153745 · Jun 6, 2011
Continuation In Part 14144413 · Dec 30, 2013
Continuation 14064136 · Oct 26, 2013
Continuation In Part 13153745 · Jun 6, 2011
Provisional Application 62637567 · Mar 2, 2018
Provisional Application 62611780 · Dec 29, 2017
Provisional Application 62593440 · Dec 1, 2017
Provisional Application 62593449 · Dec 1, 2017
Provisional Application 62557460 · Sep 12, 2017
Provisional Application 62541847 · Aug 7, 2017
Provisional Application 62524606 · Jun 25, 2017
Provisional Application 62503485 · May 9, 2017
Provisional Application 62469591 · Mar 10, 2017
Provisional Application 62448448 · Jan 20, 2017
Provisional Application 62370421 · Aug 3, 2016
Provisional Application 62301558 · Feb 29, 2016
Provisional Application 62273896 · Dec 31, 2015
Provisional Application 62265937 · Dec 10, 2015
Provisional Application 62222518 · Sep 23, 2015
Provisional Application 62128974 · Mar 5, 2015
Provisional Application 62082579 · Nov 20, 2014
Provisional Application 62047508 · Sep 8, 2014
Provisional Application 62023800 · Jul 11, 2014
Provisional Application 61972314 · Mar 30, 2014
Provisional Application 61953878 · Mar 16, 2014
Provisional Application 61927481 · Jan 15, 2014
Provisional Application 61924252 · Jan 7, 2014
Provisional Application 61916190 · Dec 14, 2013
Provisional Application 61867007 · Aug 16, 2013
Provisional Application 61467209 · Mar 24, 2011
Provisional Application 61447464 · Feb 28, 2011
Provisional Application 61447089 · Feb 27, 2011
Provisional Application 61439913 · Feb 6, 2011
Provisional Application 61414451 · Nov 17, 2010
Provisional Application 61388002 · Sep 30, 2010
Provisional Application 61352166 · Jun 7, 2010
Provisional Application 61844478 · Jul 10, 2013
Provisional Application 61789038 · Mar 15, 2013
Provisional Application 61790461 · Mar 15, 2013
Provisional Application 61793761 · Mar 15, 2013
Provisional Application 61798731 · Mar 15, 2013
Provisional Application 61747651 · Dec 31, 2012
Provisional Application 61747810 · Dec 31, 2012
Provisional Application 61719383 · Oct 27, 2012
Provisional Application 62625274 · Feb 1, 2018
Related Publication 20180196432A1 · Jul 12, 2018