Emotion, mood and personality inference in real-time environments
View Patent ↗Methods and systems monitor communications between users and analyze the communications to simultaneously determine, for a current time period, mental state variables of one of the users. Such mental state variables include the emotion of the user, the mood of the user, and the personality of the user. Additionally, such methods aggregate the emotion, the mood, and the personality using a hierarchical probabilistic graphical model that determines the highest probability path through a directed probabilistic graph to infer the mental state of the user. The directed probabilistic graph maintains a single state for the personality for the time period, and maintains multiple states for the emotion and the mood for the time period. These methods and systems output the mental state of the user.
1. A method comprising:
automatically monitoring communications using a specialized language processor, said communications comprising computerized text forming a conversation conducted between at least two conversation partners, said conversation partners comprising an agent using a specialized language processor and a user, said conversation comprising multiple turns of text utterances exchanged between said conversation partners relating to a specific issue;
automatically analyzing said communications using said specialized language processor to simultaneously determine, for said conversation, mental state variables of said user, said mental state variables comprising:
an emotion of said user;
a mood of said user; and
a personality of said user; and
automatically aggregating said emotion, said mood, and said personality using a hierarchical probabilistic graphical model that determines a highest probability path through a directed probabilistic graph to infer said mental state of said user, using said specialized language processor, said directed probabilistic graph comprising a single personality node that maintains a single unchanging state, multiple mood nodes, multiple emotion nodes, and multiple evidence nodes, and each of said mood nodes, said emotion nodes, and said evidence nodes being for a different time portion of said conversation;
automatically and constantly updating said mental state of said user output from said specialized language processor during said conversation by maintaining, in said directed probabilistic graph, a single unchanging state for said personality for all of said conversation, and maintaining multiple changing states for said emotion and said mood as said conversation progresses to track said mental state of said user during said conversation;
automatically and constantly updating said mental state of said user output from said specialized language processor as said specialized language processor tracks said mental state of said user during said conversation; and
automatically and constantly displaying said mental state to said agent through a graphic user interface as said mental state is constantly updated during said conversation.
2. The method according to claim 1 , said directed probabilistic graph comprising edges connecting said personality node, said mood nodes, said emotion nodes, and said evidence nodes, and
said edges comprising probability values.
3. The method according to claim 2 , a path through said directed probabilistic graph comprising a series of adjacent nodes,
a probability of said path comprising an aggregation of said probabilities of said edges of said series of adjacent nodes, and
said highest probability path having an aggregation of said probabilities of said edges that is higher than all other possible paths through said directed probabilistic graph.
4. The method according to claim 1 , said evidence nodes comprising different dialogue variables used by said personality node, said mood nodes, and said emotion nodes.
5. The method according to claim 1 , said emotion comprising happy-for-satisfaction, anger, or distress,
said mood comprising positive, neutral, or negative, and
said personality comprising neuroticism, extraversion, openness to experience, agreeableness, or conscientiousness.
6. The method according to claim 1 , said single personality node being stable over said conversation, said multiple mood nodes having short term temporal stability over said conversation, and said multiple emotion nodes having instantaneous temporal stability over said conversation.
7. A method comprising:
automatically monitoring communications using a specialized language processor, said communications comprising computerized text forming a conversation conducted between at least two conversation partners, said conversation partners comprising an agent using a specialized language processor and a user, said conversation comprising multiple turns of text utterances exchanged between said conversation partners relating to a specific issue;
automatically analyzing said text utterances using said specialized language processor to simultaneously determine, for said conversation, mental state variables of said user, said mental state variables comprising:
an emotion of said user;
a mood of said user; and
a personality of said user; and
automatically aggregating said emotion, said mood, and said personality using a hierarchical probabilistic graphical model that determines a highest probability path through a directed probabilistic graph to infer said mental state of said user, using said specialized language processor, said directed probabilistic graph comprising a single personality node that maintains a single unchanging state, multiple mood nodes, multiple emotion nodes, and multiple evidence nodes, and each of said mood nodes, said emotion nodes, and said evidence nodes being for a different time portion of said conversation; and
automatically and constantly updating said mental state of said user output from said specialized language processor during said conversation by maintaining, in said directed probabilistic graph, a single unchanging state for said personality for all of said conversation, and maintaining multiple changing states for said emotion and said mood as said conversation progresses to track said mental state of said user during said conversation; and
automatically and constantly updating said mental state of said user through said graphic user interface as said specialized language processor tracks said mental state of said user during said conversation; and
automatically and constantly displaying said mental state to said agent through a graphic user interface as said mental state is constantly updated during said conversation by displaying said emotion, said mood, and said personality on said graphic user interface.
8. The method according to claim 7 , said directed probabilistic graph comprising edges connecting said personality node, said mood nodes, said emotion nodes, and said evidence nodes, and
said edges comprising probability values.
9. The method according to claim 8 , a path through said directed probabilistic graph comprising a series of adjacent nodes,
a probability of said path comprising an aggregation of said probabilities of said edges of said series of adjacent nodes, and
said highest probability path having an aggregation of said probabilities of said edges that is higher than all other possible paths through said directed probabilistic graph.
10. The method according to claim 7 , said evidence nodes comprising different dialogue variables used by said personality node, said mood nodes, and said emotion nodes.
11. The method according to claim 7 , said emotion comprising happy-for-satisfaction, anger, or distress,
said mood comprising positive, neutral, or negative, and
said personality comprising neuroticism, extraversion, openness to experience, agreeableness, or conscientiousness.
12. The method according to claim 7 , said single personality node being stable over said conversation, said multiple mood nodes having short term temporal stability over said conversation, and said multiple emotion nodes having instantaneous temporal stability over said conversation.
13. A system comprising:
a specialized language processor; and
a graphic user interface operatively connected to said specialized language processor,
said specialized language processor automatically monitoring communications, said communications comprising computerized text forming a conversation conducted between at least two conversation partners, said conversation partners comprising an agent using a specialized language processor and a user, said conversation comprising multiple turns of text utterances exchanged between said conversation partners relating to a specific issue;
said specialized language processor automatically analyzing said communications to simultaneously determine, for said conversation, mental state variables of said user, said mental state variables comprising:
an emotion of said user;
a mood of said user; and
a personality of said user; and
said specialized language processor automatically aggregating said emotion, said mood, and said personality using a hierarchical probabilistic graphical model that determines a highest probability path through a directed probabilistic graph to infer said mental state of said user,
said directed probabilistic graph comprising a single personality node that maintains a single unchanging state, multiple mood nodes, multiple emotion nodes, and multiple evidence nodes,
each of said mood nodes, said emotion nodes, and said evidence nodes being for a different time portion of said conversation,
said specialized language processor automatically and constantly updating said mental state of said user during said conversation by maintaining, in said directed probabilistic graph, a single unchanging state for said personality for all of said conversation, and maintaining multiple changing states for said emotion and said mood as said conversation progresses to track said mental state of said user during said conversation,
said specialized language processor automatically and constantly updating said mental state of said user as said specialized language processor tracks said mental state of said user during said conversation, and
said graphic user interface automatically and constantly displaying said mental state to said agent as said mental state is constantly updated during said conversation by displaying said emotion, said mood, and said personality.
14. The system according to claim 13 , said directed probabilistic graph comprising edges connecting said personality node, said mood nodes, said emotion nodes, and said evidence nodes, and
said edges comprising probability values.
15. The system according to claim 14 , a path through said directed probabilistic graph comprising a series of adjacent nodes,
a probability of said path comprising an aggregation of said probabilities of said edges of said series of adjacent nodes, and
said highest probability path having an aggregation of said probabilities of said edges that is higher than all other possible paths through said directed probabilistic graph.
16. The system according to claim 14 , said evidence nodes comprising different dialogue variables used by said personality node, said mood nodes, and said emotion nodes.
17. The system according to claim 13 , said emotion comprising happy-for-satisfaction, anger, or distress,
said mood comprising positive, neutral, or negative, and
said personality comprising neuroticism, extraversion, openness to experience, agreeableness, or conscientiousness.
18. The system according to claim 13 , said single personality node being stable over said conversation, said multiple mood nodes having short term temporal stability over said conversation, and said multiple emotion nodes having instantaneous temporal stability over said conversation.