Systems and methods to enhance interactive engagement with shared content by a contextual virtual agent
Systems and methods are described to enhance interactive engagement during simultaneous delivery of serial or digital content (e.g., audio, video) to a plurality of users. A machine-based awareness of the context of the content and/or one or more user reactions to the presentation of the content may be used as a basis to interrupt content delivery in order to intersperse a snippet that includes a virtual agent with an awareness of the context(s) of the content and/or the one or more user reactions. This “contextual virtual agent” (CVA) enacts actions and/or dialog based on the one or more machine-classified contexts coupled with identified interests and/or aspirations of individuals within the group of users. The CVA may also base its activities on a machine-based awareness of “future” content that has not yet been delivered to the group, but classified by natural language and/or computer vision processing. Interrupting the delivery of content substantially simultaneously to a group of users and initiating dialog regarding content by a CVA enhances opportunities for users to engage with each other about their shared interactive experience.
1. A method to encourage human engagement, comprising:
providing a plurality of electronic devices, each electronic device comprising a processor, an output device operatively coupled to the processor, and a sensor operatively coupled to the processor for monitoring reactions of a user of the electronic device;
delivering, substantially simultaneously on output devices of all of the electronic devices, serial content for users of all of the electronic devices to view as a group;
detecting, by one or more of one or more processors of the electronic devices and one or more sensors of the electronic devices, a pause indication related to emotional or facial reactions of one or more of the users to the serial content;
pausing, substantially simultaneously on all of the output devices, the delivering of the serial content based on the pause indication;
determining, by the one or more processors, one or more serial content contexts related to the serial content coincident with the pausing of the delivering of the serial content; and
generating a virtual agent as a displayed character with voice that is presented substantially simultaneously on all of the output devices using the one or more processors, the virtual agent initiating an interaction including a conversation between the virtual agent and one or more of the users based on the one or more serial content contexts.
2. The method of claim 1 , wherein each of the electronic devices comprises one or more of one or more tablet devices, mobile phones, laptop computers, desktop computers, gaming devices, monitors, televisions, smart displays, speakers, smart speakers, projection devices, tactile electronic displays, virtual reality headsets, augmented reality headwear, and holographic displays.
3. The method of claim 1 , wherein the serial content comprises one or more of audiovisual data, a video stream, a movie, an audio recording, a plurality of images, a multimedia presentation, a podcast, an audio book, output produced by an application, and an animation.
4. The method of claim 1 , wherein the serial content context is determined by one or more of acquiring context labelling of one or more segments of the serial content, classifying the serial content by natural language processing, and classifying the serial content by image recognition processing.
5. The method of claim 1 , wherein the virtual agent is generated as one or more of one or more displayed cartoon characters, displayed animals, displayed persons, displayed avatars, displayed icons, projected holograms, robots, and animated toys delivered simultaneously on all of the output devices.
6. The method of claim 1 , wherein the processor is instantiated with an artificial intelligence.
7. The method of claim 1 , further comprising, upon detecting the pause indication, determining, by the one or more processors, one or more ensuing serial content contexts after pausing the displaying of the serial content.
8. The method of claim 1 , further comprising:
acquiring, from the interaction with the one or more humans, interaction data from the sensor of at least one of the electronic devices;
classifying from the interaction data, using one or more processors, one or more content reactions by the one or more humans; and
initiating, by the virtual agent generated on all of the output devices using the one or more processors, one or more additional interactions with the one or more humans based on the one or more content reactions.
9. The method of claim 1 , wherein the users comprise a child and one or more adults and wherein the virtual agent enhances interactive engagement between the child and the one or more adults.
10. The method of claim 1 , wherein the interaction by the virtual agent prompts an exchange between the users related to the serial content.
11. The method of claim 1 , wherein the one or more sensors comprise one or more cameras that are used to identify facial expressions or gestures of one or more of the users to identify the reactions.
12. A method to encourage human engagement, comprising:
providing a plurality of electronic devices, each electronic device comprising a processor, and an output device operatively coupled to the processor;
delivering, substantially simultaneously on output devices of all of the electronic devices, serial content for users of all of the electronic devices to view as a group;
determining, by the one or more processors, one or more serial content contexts, wherein the one or more serial content contexts comprise serial content within the serial content that generates one or more human feelings or reactions;
determining, by one or more processors, that the one or more serial content contexts match one or more predetermined contexts;
pausing, substantially simultaneously on all of the output devices, the delivering of the serial content; and
generating a virtual agent as a displayed character with voice that is presented substantially simultaneously on all of the output devices using the one or more processors, the virtual agent initiating an interaction including a conversation between the virtual agent and one or more of the users based on the one or more serial content contexts.
13. The method of claim 12 , wherein each of the electronic devices comprises one or more of one or more tablet devices, mobile phones, laptop computers, desktop computers, gaming devices, monitors, televisions, smart displays, speakers, smart speakers, projection devices, tactile electronic displays, virtual reality headsets, augmented reality headwear, and holographic displays.
14. The method of claim 12 , wherein the serial content comprises one or more of audiovisual data, a video stream, a movie, an audio recording, a plurality of images, a podcast, an audio book, output produced by an application, and an animation.
15. The method of claim 12 , wherein the serial content context is determined by one or more of acquiring context labelling of one or more segments of the serial content, classifying the serial content by natural language processing, and classifying the serial content by image recognition processing.
16. The method of claim 12 , wherein the one or more serial content contexts comprise content within the serial content that generates one or more human feelings of one or more of surprise, amusement, fear, horror, anger, rage, disgust, annoyed, contempt, sadness, joy, confusion, interest, boredom, calmness, anxiety, anticipation, envy, sexual desire, love, and friendship.
17. The method of claim 12 , wherein the virtual agent is generated as one or more of one or more displayed cartoon characters, displayed animals, displayed persons, displayed avatars, displayed icons, projected holograms, robots, and animated toys delivered simultaneously on all of the output devices.
18. The method of claim 12 , wherein the processor is instantiated with an artificial intelligence.
19. The method of claim 12 , further comprising, upon determining that the one or more serial content contexts match one or more predetermined contexts, determining by the one or more processors, one or more ensuing serial content contexts after pausing the displaying of the serial content.
20. The method of claim 12 , further comprising:
acquiring from the interaction with the one or more humans, interaction data from one or more sensors;
classifying from the interaction data, using one or more processors, one or more content reactions by the one or more humans; and
initiating, by the virtual agent generated on the output devices using the one or more processors, one or more additional interactions with the one or more humans based on the one or more content reactions.
21. The method of claim 12 , wherein the one or more serial content contexts comprise scenes or events within the serial content that generate one or more human feelings or reactions and wherein the conversation relates to the one or more human feelings or reactions.
22. A system to encourage human engagement, comprising:
a plurality of electronic devices, each electronic device comprising a processor, an output device operatively coupled to the processor, and a sensor operatively coupled to the processor for monitoring reactions of a user of the electronic device,
wherein the electronic devices are configured for:
delivering, substantially simultaneously on output devices of all of the electronic devices, serial content for users of all of the electronic devices to view as a group;
detecting, by one or more of one or more processors of the electronic devices and one or more sensors of the electronic devices, a pause indication related to emotional or facial reactions of one or more of the users to the serial content;
pausing, substantially simultaneously on all of the output devices, the delivering of the serial content based on the pause indication;
determining, by the one or more processors, one or more serial content contexts related to the serial content coincident with the pausing of the delivering of the serial content; and
generating a virtual agent as a displayed character with voice that is presented substantially simultaneously on all of the output devices using the one or more processors, the virtual agent initiating an interaction including a conversation between the virtual agent and one or more of the users based on the one or more serial content contexts.