Adjusting chat bot design based on matched escalation logs
An example system includes a processor to receive a bot design and escalation logs associated with a chat bot configured based on the bot design. The processor can compute a similarity score between each of a number of bot response nodes in the bot design and the escalation logs. The processor can generate a sorted list of the bot response nodes in the bot design based on the similarity scores.
1. A system, comprising a processor to:
receive a bot design and escalation logs associated with a chat bot configured based on the bot design;
compute a similarity score between each bot response node of a plurality of bot response nodes in the bot design and each escalation log of the escalation logs;
generate a sorted list of the plurality of bot response nodes in the bot design based on the similarity scores of the plurality of bot response nodes in the bot design and the escalation logs;
tune, in response to the similarity score indicating a failure of the chat bot, an intent detection component of the chat bot to correctly handle a user query by transferring a user to a correct bot response node of the plurality of bot response nodes in the bot design in response to adjusting the bot design to incorporate new topics identified from clustered escalation logs and to refine existing user intents based on flagged bot response nodes of the plurality of bot response nodes in the bot design; and
execute the chat bot with the adjusted bot design to successfully complete a conversation with the user without escalation.
2. The system of claim 1 , wherein the processor is further to flag a bot response node in the bot design in response to detecting that the similarity score between a bot response node and an escalation log exceeds a threshold score.
3. The system of claim 2 , wherein the processor is further to cluster the escalation logs based on topic and modify the bot design to include a generated topic in response to detecting that a cluster of escalation logs does not exceed another threshold score lower than the threshold score and detecting that the generated topic is not out-of-scope for the bot design.
4. The system of claim 1 , wherein the processor is further to filter out fluff utterances via a classifier trained on training data associated with the bot design.
5. The system of claim 1 , wherein the processor is further to select portions of the escalation logs for computing the similarity score comprising filtered agent side utterances and a first user utterance to an agent.
6. The system of claim 1 , wherein the processor is further to compute an accumulative similarity score for each bot response node of the plurality of bot response nodes in the bot design by summing together pairwise similarity scores computed between each bot response node of the plurality of bot response nodes in the bot design and each escalation log of the escalation logs.
7. The system of claim 1 , wherein the similarity score comprises a pairwise calculated semantic similarity score between each bot response node of the plurality of bot response nodes in the bot design and each escalation log of the escalation logs.
8. A computer-implemented method, comprising:
receiving, via a processor, a bot design and escalation logs associated with a chat bot configured based on the bot design;
selecting, via the processor, sections of conversations in the escalation logs for matching;
computing, via the processor, a similarity score between each bot response node of a plurality of bot response nodes in the bot design and each escalation log of the escalation logs;
sorting, via the processor, the plurality of bot response nodes in the bot design by the similarity scores of the plurality of bot response nodes in the bot design and the escalation logs to generate a sorted list of the plurality of bot response nodes in the bot design;
tuning, via the processor and in response to the similarity score indicating a failure of the chat bot, an intent detection component of the chat bot to correctly handle a user query by transferring a user to a correct bot response node of the plurality of bot response nodes in the bot design in response to adjusting the bot design to incorporate new topics identified from clustered escalation logs and to refine existing user intents based on flagged bot response nodes of the plurality of bot response nodes in the bot design; and
executing, via the processor, the chat bot with the adjusted bot design to successfully complete a conversation with the user without escalation.
9. The computer-implemented method of claim 8 , further comprising filtering, via the processor, fluff utterances from the conversations in the escalation logs.
10. The computer-implemented method of claim 8 , wherein selecting the sections of conversations in the escalation logs comprises selecting a first user utterance and agent utterances.
11. The computer-implemented method of claim 8 , further comprising selecting, via the processor, a portion of bot response nodes in the plurality of bot response nodes in the bot design for matching.
12. The computer-implemented method of claim 8 , wherein computing the similarity score comprises computing a full semantic similarity between each bot response node of the plurality of bot response nodes in the bot design and the escalation logs.
13. The computer-implemented method of claim 8 , wherein computing the similarity score comprises computing a conversation to first-node-sentence semantic similarity.
14. The computer-implemented method of claim 8 , wherein computing the similarity score comprises computing a fusion of a full semantic similarity and a conversation to first-node-sentence semantic similarity between each bot response node of the plurality of bot response nodes in the bot design and the escalation logs.
15. The computer-implemented method of claim 8 , further comprising flagging, via the processor, a bot response node for modification in response to detecting that the similarity scores of the plurality of bot response nodes in the bot design exceeds a threshold score.
16. The computer-implemented method of claim 8 , further comprising generating, via the processor, a new topic to be added to the bot design in response to detecting that a cluster of escalation logs of a same topic have a semantic similarity score not exceeding a threshold score for any bot response node of the plurality of bot response nodes in the bot design and that the new topic is not out-of-scope for the bot design.
17. The computer-implemented method of claim 8 , further comprising adjusting, via the processor, the bot design to identify a user request as out-of-scope for the bot design in response to detecting that a cluster of escalation logs of a same topic have a semantic similarity score not exceeding a threshold score for any bot response node of the plurality of bot response nodes in the bot design and that a new topic is out-of-scope for the bot design.
18. A computer program product for sorting bot response nodes, the computer program product comprising a computer readable storage medium having program code embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program code executable by a processor to cause the processor to:
receive a bot design and escalation logs associated with a chat bot configured based on the bot design;
select sections of conversations in the escalation logs for matching;
compute a similarity score between each bot response node of a plurality of bot response nodes in the bot design and each escalation log of the escalation logs;
sort the plurality of bot response nodes in the bot design by the similarity scores of the plurality of bot response nodes in the bot design and the escalation logs to generate a sorted list of the plurality of bot response nodes in the bot design;
tune, in response to the similarity score indicating a failure of the chat bot, an intent detection component of the chat bot to correctly handle a user query by transferring a user to a correct bot response node of the plurality of bot response nodes in the bot design in response to adjusting the bot design to incorporate new topics identified from clustered escalation logs and to refine existing user intents based on flagged bot response nodes of the plurality of bot response nodes in the bot design; and
execute the chat bot with the adjusted bot design to successfully complete a conversation with the user without escalation.
19. The computer program product of claim 18 , further comprising program code executable by the processor to cause the processor to flag a bot response node for modification in response to detecting that the similarity score of the plurality of bot response nodes in the bot design exceeds a threshold score.
20. The computer program product of claim 18 , further comprising program code executable by the processor to generate a new topic to be added to the bot design in response to detecting that a cluster of escalation logs of a same topic have a semantic similarity score not exceeding a threshold score for any bot response node of the plurality of bot response nodes in the bot design and that the new topic is not out-of-scope for the bot design.