Multidevice multimodal emotion services monitoring
Techniques for multidevice, multimodal emotion services monitoring are disclosed. An expression to be detected is determined. The expression relates to a cognitive state of an individual. Input on the cognitive state of the individual is obtained using a device local to the individual. Monitoring for the expression is performed. The monitoring uses a background process on a device remote from the individual. An occurrence of the expression is identified. The identification is performed by the background process. Notification that the expression was identified is provided. The notification is provided from the background process to a device distinct from the device running the background process. The expression is defined as a multimodal expression. The multimodal expression includes image data and audio data from the individual. The notification enables emotion services to be provided. The emotion services augment messaging, social media, and automated help applications.
1. A computer-implemented method for multi-device analysis comprising:
determining an expression to be detected, wherein the expression relates to a cognitive state of an individual;
obtaining input on the cognitive state of the individual using a first device local to the individual;
monitoring for the expression, using a background process on a second device remote from the individual;
identifying an occurrence of the expression, by the background process;
providing notification, to a foreground application, that the expression was identified, wherein the notification is provided from the background process to a third device, wherein the third device is distinct from the second device, wherein the foreground application comprises a social media application;
enabling emotion services using two or more devices; and
performing an emotion response action in response to the notification that was provided, wherein the emotion response action includes posting a message on a social media system.
2. The method of claim 1 wherein the expression is defined as a multimodal expression.
3. The method of claim 2 wherein the multimodal expression includes image data and audio data from the individual.
4. The method of claim 1 wherein the providing notification enables the emotion services.
5. The method of claim 4 wherein the emotion services are requested by the individual.
6. The method of claim 4 wherein the emotion services are requested by a third party.
7. The method of claim 1 wherein the first device and the third device are the same device.
8. The method of claim 1 wherein the background process is executed using a neural network.
9. The method of claim 1 wherein an application requests the notification.
10. The method of claim 9 wherein the application is running on the third device.
11. The method of claim 9 wherein the application is running on a fourth device.
12. The method of claim 1 wherein a camera sensor is on continuously as part of the monitoring.
13. The method of claim 1 wherein the providing includes delivering a coupon or a reward.
14. The method of claim 13 wherein the providing is part of a gaming experience.
15. The method of claim 1 wherein the expression is a reflection of a cognitive state.
16. The method of claim 1 wherein the identifying is provided with a confidence interval.
17. The method of claim 1 wherein the monitoring is accomplished passively.
18. The method of claim 1 wherein the identifying is enabled by inputting an expression description.
19. The method of claim 1 wherein the identifying is enabled by providing an expression contained in an image.
20. The method of claim 1 wherein the notification is provided using an expression contained in an image.
21. The method of claim 20 wherein the foreground application further comprises a messaging application configured to send messages between a sender and a recipient, and wherein, upon the recipient accessing a first message, the background process identifies an occurrence of the expression that was identified, and wherein the emotion response action includes sending a second message to the sender which includes emotion scoring, wherein the second message is indicative of an emotional response of the recipient in response to the first message.
22. The method of claim 21 wherein the second message further includes the image.
23. The method of claim 21 wherein the second message further includes an emoji.
24. The method of claim 20 wherein the social media application is configured to display a first message, and wherein, upon a recipient accessing the first message, the background process identifies an occurrence of the expression that was identified, and wherein the posting the message on the social media system includes emotion scoring, wherein the message, on the social media system that was posted, is indicative of an emotional response of the recipient in response to the first message.
25. The method of claim 20 wherein the social media application is configured to display a message, and wherein, upon a recipient accessing the message, the background process identifies an occurrence of the expression that was identified, and wherein the emotion response action includes generating an automatic like of the message.
26. The method of claim 20 wherein the emotion response action includes invoking an automated help application.
27. A computer program product embodied in a non-transitory computer readable medium for multi-device analysis, the computer program product comprising code which causes one or more processors to perform operations of:
determining an expression to be detected, wherein the expression relates to a cognitive state of an individual;
obtaining input on the cognitive state of the individual using a first device local to the individual;
monitoring for the expression, using a background process on a second device remote from the individual;
identifying an occurrence of the expression, by the background process;
providing notification, to a foreground application, that the expression was identified, wherein the notification is provided from the background process to a third device, wherein the third device is distinct from the second device, wherein the foreground application comprises a social media application;
enabling emotion services using two or more devices; and
performing an emotion response action in response to the notification that was provided, wherein the emotion response action includes posting a message on a social media system.
28. A computer system for multi-device analysis comprising:
a memory which stores instructions;
one or more processors coupled to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to:
determine an expression to be detected, wherein the expression relates to a cognitive state of an individual;
obtain input on the cognitive state of the individual using a first device local to the individual;
monitor for the expression, using a background process on a second device remote from the individual;
identify an occurrence of the expression, by the background process;
provide notification, to a foreground application, that the expression was identified, wherein the notification is provided from the background process to a third device, wherein the third device is distinct from the second device, wherein the foreground application comprises a social media application;
enable emotion services using two or more devices; and
perform an emotion response action in response to the notification that was provided, wherein the emotion response action includes posting a message on a social media system.