Granted Patent B2

US 10,346,542 · App. 13/778,824 · Granted Jul 9, 2019

Inventor: Charles C Wooters (Annapolis, MD)

Assignee: VERINT AMERICAS INC.

G06F17/28G06F17/279

Abstract

Claims (106)

1. A method comprising:

accessing one or more log files of a corpus of human-to-human conversations;

performing textual analysis on the one or more log files, the textual analysis including identifying carriage returns, tabs, changes in font, and/or changes in text color;

based at least in part on the textual analysis, converting each of the one or more log files into a different file format that indicates dialogue turns and an individual associated with each dialogue turn, each dialogue turn representing input from the respective individual;

identifying, from the dialogue turns of a first conversation in the corpus, a particular dialog turn that includes a question from a particular individual;

representing the dialog turns of the particular individual as a series of indicia, positions in the series including either a first indicia indicating that a corresponding dialogue turn includes a question or a second indicia indicating that a corresponding dialog turn does not include a question;

comparing the series of indicia to a predefined sequence of indicia;

determining, based at least in part on comparing the series of indicia to the predefined sequence of indicia, that the first conversation has workflow-like features;

determining, using one or more processing units, a first metric and a second metric for the first conversation in the corpus, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have the workflow-like features, the work-flow like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining, using the one or more processing units, a first metric and a second metric for a second conversation in the corpus, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in creating a virtual agent to replace a human associated with the corpus; and

displaying, via a graphical user interface, information that is based at least in part on the representative value.

2. The method of claim 1 , wherein, the corpus comprises over 1,000 individual conversations.

3. The method of claim 1 , wherein at least one of the first conversation or the second conversation includes a conversation between a customer who asks one or more questions and a human agent who answers the one or more questions.

4. The method of claim 3 , further comprising providing a suggestion to replace the human agent with the virtual agent.

5. The method of claim 1 , wherein at least one of the first conversation or the second conversation is stored in the corpus as a live text chat between two humans.

6. The method of claim 1 , wherein at least one of the first conversation or the second conversation is stored in the corpus as a text record generated from speech-to-text analysis of spoken communication between two humans.

7. The method of claim 1 , wherein the determining the first metric for the first conversation includes determining at least one of:

a number of separate questions contained in a dialogue turn of a customer for the first conversation;

a length of a communication in the dialogue turn of the customer for the first conversation;

a number of dialogue turns in the first conversation;

a number of dialogue turns of an agent for the first conversation that contain at least one question; or

a semantic distance between dialogue turns for the first conversation.

8. The method of claim 1 , wherein the determining the second metric for the first conversation includes determining at least one of:

a number and order of dialogue turns of an agent for the first conversation that contain at least one question; or

a number and order of dialogue turns of the agent for the first conversation that do not include a question.

9. The method of claim 1 , further comprising

providing, via one or more graphical user interfaces, a first metric indicating a first measure of the corpus of human-to-human conversations for which a human could be replaced with a virtual agent; and

providing, via the one or more graphical user interfaces, a second metric indicating a second measure of the corpus of human-to-human conversations for which a human could not be replaced with a virtual agent.

10. One or more non-transitory computer-readable storage media, configured to store computer-executable instructions that, when executed on one or more processors, cause the one or more processors to perform acts comprising:

obtaining a recording of a spoken communication between a first individual and a second individual;

performing speaker recognition on the recording to identify first audio characteristics of the first individual and second audio characteristics of the second individual;

identifying, based at least in part on the first audio characteristics, first dialogue turns representing first input from the first individual;

identifying, based at least in part on the second audio characteristics, second dialog turns representing second input from the second individual;

generating one or more log files including indications of the first dialog turns and the second dialog turns;

storing the one or more log files as a first conversation in a corpus of human-to-human conversations;

determining a first metric and a second metric for the first conversation in the corpus, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have workflow-like features, the workflow-like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining a first metric and a second metric for a second conversation in the corpus, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in replacing a human associated with the corpus with a virtual agent; and

providing, via a graphical user interface, information that is based at least in part on the representative value.

11. The one or more non-transitory computer-readable storage media of claim 10 , wherein the determining the first metric for the first conversation includes determining a number of separate questions contained in a dialogue turn of a customer for the first conversation based on at least one of a number of question marks in the dialogue turn or natural language processing.

12. The one or more non-transitory computer-readable storage media of claim 10 , wherein the determining the first metric for the first conversation includes determining a length of a dialogue turn of a customer for the first conversation based on at least one of a number of words of the dialogue turn, a number of characters in the dialogue turn, or a length of time of the dialogue turn.

13. The one or more non-transitory computer-readable storage media of claim 10 , wherein the determining the first metric for the first conversation includes determining a number of dialogue turns in the first conversation.

14. The one or more non-transitory computer-readable storage media of claim 10 , wherein the determining the first metric for the first conversation includes determining a number of dialogue turns of an agent for the first conversation that contain at least one question.

15. The one or more non-transitory computer-readable storage media of claim 10 , wherein the determining the first metric for the first conversation includes determining a semantic distance between dialogue turns for the first conversation.

16. The one or more non-transitory computer-readable storage media of claim 10 , wherein the determining the second metric for the first conversation includes determining a number and order of dialogue turns of an agent for the first conversation that contain at least one question.

17. The one or more non-transitory computer-readable storage media of claim 10 , wherein the determining the second metric for the first conversation includes determining a number and order of dialogue turns of an agent for the first conversation that do not include a question.

18. A system comprising:

one or more processors; and

memory storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

obtaining first data representing a first conversation, the first data being in a first data format;

obtaining second data representing a second conversation, the second data being in a second data format that is different than the first format;

converting the first data from the first data format to a third data format;

converting the second data from the second data format to the third data format, wherein the computer-executable instructions are configured to perform textual analysis on data in the third data format;

performing textual analysis on the first data of the first conversation and the second data of the second conversation;

based at least in part on the textual analysis, identifying dialogue turns and an individual associated with each dialogue turn, each dialogue turn representing input from the respective individual;

determining a first metric and a second metric for the first conversation, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have workflow-like features, the workflow-like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining a first metric and a second metric for the second conversation, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in replacing a human associated with the first conversation or the second conversation with a virtual agent; and

providing, via a graphical user interface, information that is based at least in part on the representative value.

19. The system of claim 18 , wherein the determining the first metric for the first conversation is based at least in part on a number of separate questions contained in a dialogue turn of a participant for the first conversation.

20. The system of claim 18 , wherein the determining the first metric for the first conversation is based at least in part on a length of a dialogue turn of a participant for the first conversation.

21. The system of claim 18 , wherein the determining the first metric for the first conversation is based at least in part on a number of dialogue turns in the first conversation.

22. The system of claim 18 , wherein the determining the first metric for the first conversation is based at least in part on a number of dialogue turns of a participant for the first conversation that contain at least one question.

23. The system of claim 18 , wherein the determining the first metric for the first conversation is based at least in part on a semantic distance between dialogue turns for the first conversation.

24. The system of claim 18 , wherein the determining the second metric for the first conversation is based at least in part on a number and order of dialogue turns of a participant for the first conversation that contain at least one question.

25. The system of claim 18 , wherein the determining the second metric for the first conversation is based at least in part on a number and order of dialogue turns of a participant for the first conversation that do not include a question.

26. A system comprising:

one or more processors; and

memory storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

performing textual analysis on data of a first conversation and data of a second conversation;

based at least in part on the textual analysis, identifying dialogue turns and an individual associated with each dialogue turn, each dialogue turn representing input from the respective individual;

determining a first metric and a second metric for the first conversation, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have workflow-like features, the workflow-like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining a first metric and a second metric for the second conversation, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in replacing a human with a virtual agent; and

providing, via a graphical user interface, information that is based at least in part on the representative value.

27. The system of claim 26 , wherein the determining the first metric for the first conversation is based on at least one of:

a number of separate questions contained in a dialogue turn for the first conversation;

a length of a communication in the dialogue turn for the first conversation;

a number of dialogue turns in the first conversation;

a number of dialogue turns for the first conversation that contain at least one question; or

a semantic distance between dialogue turns for the first conversation.

28. The system of claim 26 , wherein the determining the second metric for the first conversation includes determining at least one of:

a number and order of dialogue turns for the first conversation that contain at least one question; or

a number and order of dialogue turns for the first conversation that do not include a question.

29. The system of claim 26 , wherein the operations further comprise:

based at least in part on the representative value, providing a suggestion to replace the human with the virtual agent.

30. A method comprising:

accessing one or more log files of a corpus of human-to-human conversations;

performing textual analysis on the one or more log files, the textual analysis including identifying carriage returns, tabs, changes in font, and/or changes in text color;

based at least in part on the textual analysis, converting each of the one or more log files into a different file format that indicates dialogue turns and an individual associated with each dialogue turn, each dialogue turn representing input from the respective individual;

determining, using one or more processing units, a first metric and a second metric for a first conversation in the corpus, the first metric for the first conversation indicating a complexity of the first conversation and the second metric for the first conversation indicating a degree to which the first conversation is deemed to have workflow-like features, the work-flow like features indicating that at least a portion of the first conversation comprises an algorithmic series of communications between two humans to accomplish a task;

determining a score for the first conversation based at least in part on the first metric for the first conversation and the second metric for the first conversation;

determining, using the one or more processing units, a first metric and a second metric for a second conversation in the corpus, the first metric for the second conversation indicating a complexity of the second conversation and the second metric for the second conversation indicating a degree to which the second conversation is deemed to have workflow-like features;

determining a score for the second conversation based at least in part on the first metric for the second conversation and the second metric for the second conversation;

combining the score for the first conversation and the score for the second conversation to generate a representative value, the representative value being indicative of difficulty in creating a virtual agent to replace a human associated with the corpus; and

displaying, via a graphical user interface, information that is based at least in part on the representative value;

based at least in part on the representative value, providing a suggestion to replace the human with the virtual agent.

Assignments (9)

Continuity (2)

Cited By (2)

US 12,223,946 US 12,436,974