Automated telecommunication session adjustment based on identified emotional states
View Patent ↗Apparatuses, systems, and methods relate to technology to provide first content to a first computing device to be presented on the first computing device as part of a telecommunication session associated with a first use. The technology receives telecommunication data from the first computing device as part of the telecommunication session, identifies a user characteristic of the user based on the telecommunication data, where the user characteristic is associated with the first content presented on the first computing device, further where the user characteristic is one or more of a physical characteristic of the user or an audio characteristic of the user. The technology also measures an emotional response of the user based on the user characteristic, and automatically adjusts a parameter of the telecommunication session based on the emotional response.
1 . A computing system comprising:
a processor; and
a memory having a set of instructions, which when executed by the processor, cause the computing system to:
provide first content to a first computing device to be presented on the first computing device as part of a telecommunication session associated with a first user;
receive telecommunication data from the first computing device as part of the telecommunication session;
identify a user characteristic of the first user based on the telecommunication data, wherein the user characteristic is associated with the first content presented on the first computing device, wherein the user characteristic is one or more of a physical characteristic of the first user or an audio characteristic of the first user;
measure an emotional response of the first user based on the user characteristic;
automatically adjust a parameter of the telecommunication session based on the emotional response;
identify a category of the first content;
determine a level of understanding of the category by the first user;
determine one or more of second content to be presented to the first user based on the level of understanding and the user characteristic, or third content to be presented to a second user associated with the telecommunication session based on the level of understanding and the user characteristic; and
provide one or more of the second content to the first computing device, or the third content to a second computing device associated with the second user.
2 . The computing system of claim 1 , wherein to adjust the parameter of the telecommunication session, the instructions of the memory, when executed, cause the computing system to:
one or more of provide a first overlay on a first graphical user interface of the first computing device that corresponds to the second content, or provide a second overlay on a second graphical user interface of the second computing device that corresponds to the third content.
3 . The computing system of claim 1 , wherein the instructions of the memory, when executed, cause the computing system to:
generate the first content with an artificial intelligence model;
generate, with the artificial intelligence model, an output based on the emotional response; and
cause the output to be presented on the first computing device during the telecommunication session.
4 . The computing system of claim 3 , wherein the output corresponds to one or more of a digital human presented on the first computing device, or a chatbot presented on the first computing device.
5 . The computing system of claim 1 , wherein the instructions of the memory, when executed, cause the computing system to:
determine a time stamp for one or more of a moment of confusion of the first user or a moment of anxiety of the first user based on the user characteristic, wherein the one or more of the moment of confusion or the moment of anxiety is the emotional response;
identify a portion of the first content based on the time stamp; and
identify mitigation information based on the portion to mitigate the emotional response,
wherein to adjust the telecommunication session, the instructions of the memory, when executed, cause the computing system to provide the mitigation information to the first computing device as part of the telecommunication session.
6 . The computing system of claim 5 , wherein the instructions of the memory, when executed, cause the computing system to:
identify a level of understanding of the first content by the first user;
access a synonym map that associates equivalent terms with each other;
identify one or more synonyms for the portion based on the synonym map and the level of understanding; and
generate the mitigation information to include the one or more synonyms,
wherein the user characteristic is one or more of a heart rate of the first user, pupil dilation of the first user, breathing rate of the first user, temperature of the first user, blood pressure of the first user, mouth activity of the first user, head motion of the first user or vocal quality of the first user.
7 . At least one non-transitory computer readable storage medium comprising a set of instructions, which when executed by a computing system, cause the computing system to:
provide first content to a first computing device to be presented on the first computing device as part of a telecommunication session associated with a first user;
receive telecommunication data from the first computing device as part of the telecommunication session;
identify a user characteristic of the first user based on the telecommunication data, wherein the user characteristic is associated with the first content presented on the first computing device, wherein the user characteristic is one or more of a physical characteristic of the first user or an audio characteristic of the first user;
measure an emotional response of the first user based on the user characteristic;
automatically adjust a parameter of the telecommunication session based on the emotional response;
identify a category of the first content;
determine a level of understanding of the category by the first user;
determine one or more of second content to be presented to the first user based on the level of understanding and the user characteristic, or third content to be presented to a second user associated with the telecommunication session based on the level of understanding and the user characteristic; and
provide one or more of the second content to the first computing device, or the third content to a second computing device associated with the second user.
8 . The at least one non-transitory computer readable storage medium of claim 7 , wherein to adjust the parameter of the telecommunication session, the instructions, when executed, cause the computing system to:
one or more of provide a first overlay on a first graphical user interface of the first computing device that corresponds to the second content, or provide a second overlay on a second graphical user interface of the second computing device that corresponds to the third content.
9 . The at least one non-transitory computer readable storage medium of claim 7 , wherein the instructions, when executed, cause the computing system to:
generate the first content with an artificial intelligence model;
generate, with the artificial intelligence model, an output based on the emotional response; and
cause the output to be presented on the first computing device during the telecommunication session.
10 . The at least one non-transitory computer readable storage medium of claim 9 , wherein the output corresponds to one or more of a digital human presented on the first computing device, or a chatbot presented on the first computing device.
11 . The at least one non-transitory computer readable storage medium of claim 7 , wherein the instructions, when executed, cause the computing system to:
determine a time stamp for one or more of a moment of confusion of the first user or a moment of anxiety of the first user based on the user characteristic, wherein the one or more of the moment of confusion or the moment of anxiety is the emotional response;
identify a portion of the first content based on the time stamp; and
identify mitigation information based on the portion to mitigate the emotional response,
wherein to adjust the telecommunication session, the instructions, when executed, cause the computing system to provide the mitigation information to the first computing device as part of the telecommunication session.
12 . The at least one non-transitory computer readable storage medium of claim 11 , wherein the instructions, when executed, cause the computing system to:
identify a level of understanding of the first content by the first user;
access a synonym map that associates equivalent terms with each other;
identify one or more synonyms for the portion based on the synonym map and the level of understanding; and
generate the mitigation information to include the one or more synonyms,
wherein the user characteristic is one or more of a heart rate of the first user, pupil dilation of the first user, breathing rate of the first user, temperature of the first user, blood pressure of the first user, mouth activity of the first user, head motion of the first user or vocal quality of the first user.
13 . A method comprising:
providing first content to a first computing device to be presented on the first computing device as part of a telecommunication session associated with a first user;
receiving telecommunication data from the first computing device as part of the telecommunication session;
identifying a user characteristic of the first user based on the telecommunication data, wherein the user characteristic is associated with the first content presented on the first computing device, wherein the user characteristic is one or more of a physical characteristic of the first user or an audio characteristic of the first user;
measuring an emotional response of the first user based on the user characteristic;
automatically adjusting a parameter of the telecommunication session based on the emotional response;
identifying a category of the first content;
determining a level of understanding of the category by the first user;
determining one or more of second content to be presented to the first user based on the level of understanding and the user characteristic, or third content to be presented to a second user associated with the telecommunication session based on the level of understanding and the user characteristic; and
providing one or more of the second content to the first computing device, or the third content to a second computing device associated with the second user.
14 . The method of claim 13 , wherein the adjusting comprises:
one or more of providing a first overlay on a first graphical user interface of the first computing device that corresponds to the second content, or providing a second overlay on a second graphical user interface of the second computing device that corresponds to the third content.
15 . The method of claim 13 , comprising:
generating the first content with an artificial intelligence model;
generating, with the artificial intelligence model, an output based on the emotional response; and
causing the output to be presented on the first computing device during the telecommunication session.
16 . The method of claim 15 , wherein the output corresponds to one or more of a digital human presented on the first computing device, or a chatbot presented on the first computing device.
17 . The method of claim 13 , comprising:
determining a time stamp for one or more of a moment of confusion of the first user or a moment of anxiety of the first user based on the user characteristic, wherein the one or more of the moment of confusion or the moment of anxiety is the emotional response;
identifying a portion of the first content based on the time stamp; and
identifying mitigation information based on the portion to mitigate the emotional response;
identifying a level of understanding of the first content by the first user;
accessing a synonym map that associates equivalent terms with each other;
identifying one or more synonyms for the portion based on the synonym map and the level of understanding; and
generating the mitigation information to include the one or more synonyms,
wherein the user characteristic is one or more of a heart rate of the first user, pupil dilation of the first user, breathing rate of the first user, temperature of the first user, blood pressure of the first user, mouth activity of the first user, head motion of the first user or vocal quality of the first user, and
wherein the adjusting comprises providing the mitigation information to the first computing device as part of the telecommunication session.