Dynamic conversation insights using large language models
In an example embodiment, several different fine-tuned LLMs are utilized to provide a system where an administrator can request conversational log insights using natural language and be presented with insights generated by an LLM, without necessitating the passing of any personal or sensitive data to a third-party. Furthermore, the use of several fine-tuned LLMs reduces the number of input tokens needing to be submitted to any one particular LLM, overcoming a key technical limitation of LLMs.
1 . A system comprising:
at least one hardware processor; and
a computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
accessing information regarding a first conversation between a first user and a chatbot, the first conversation involving a request for analysis of a first set of historical online conversations;
generating a first prompt comprising the information regarding the first conversation, the first prompt further comprising instructions to generate instructions to retrieve and analyze historical online conversations;
submitting the first prompt to a fine-tuned conversational classification Large Language Model (LLM);
receiving, at an orchestrator, from the fine-tuned conversational classification LLM, the instructions;
retrieving, using the instructions from the fine-tuned conversational classification LLM, by the orchestrator, the first set of historical online conversations from a conversation database;
passing, by the orchestrator, the first set of historical online conversations into a fine-tuned conversational summary and translation LLM to generate summaries of the first set of historical online conversations; and
passing, by the orchestrator, the summaries of the first set of historical online conversations to a fined-tuned conversational insight LLM to generate one or more insights about the first set of historical online conversations.
2 . The system of claim 1 , wherein the first set of historical online conversations are between users and a first application in an online network.
3 . The system of claim 1 , wherein the operations further comprise:
normalizing the first set of historical online conversations.
4 . The system of claim 1 , wherein the operations further comprise:
translating the first set of historical online conversations.
5 . The system of claim 1 , wherein the fine-tuned conversational insights LLM is fine-tuned to generate one or more metrics regarding intent of conversations in a set of historical online conversations and to generate one or more graphical charts based on the generated one or more metrics.
6 . The system of claim 1 , wherein the first set of historical online conversations contain personal data that is not shared with the fined-tuned conversational insight LLM.
7 . The system of claim 1 , wherein the first conversation is broken into a first part and a second part, the first part identifying the first set of historical online conversation and the second part identifying analysis to be performed, and wherein the first part and the second part are fed to the fine-tuned conversational classification LLM in separate prompts.
8 . The system of claim 1 , wherein the insights include a summarization of aspects conveyed during conversations in the first set of historical online conversations.
9 . The system of claim 8 , wherein the insights include one or more graphical charts indicating which percentage of conversations in the first set of historical online conversations conveyed which aspects.
10 . The system of claim 9 , wherein the aspects include mood conveyed during conversations in the first set of historical online conversations.
11 . The system of claim 9 , wherein the aspects include tone conveyed during conversations in the first set of historical online conversations.
12 . A method comprising:
accessing information regarding a first conversation between a first user and a chatbot, the first conversation involving a request for analysis of a first set of historical online conversations;
generating a first prompt comprising the information regarding the first conversation, the first prompt further comprising instructions to generate instructions to retrieve and analyze historical online conversations;
submitting the first prompt to a fine-tuned conversational classification Large Language Model (LLM);
receiving, at an orchestrator, from the fine-tuned conversational classification LLM, the instructions;
retrieving, using the instructions from the fine-tuned conversational classification LLM, by the orchestrator, the first set of historical online conversations from a conversation database;
passing, by the orchestrator, the first set of historical online conversations into a fine-tuned conversational summary and translation LLM to generate summaries of the first set of historical online conversations; and
passing, by the orchestrator, the summaries of the first set of historical online conversations to a fined-tuned conversational insight LLM to generate one or more insights about the first set of historical online conversations.
13 . The method of claim 12 , wherein the first set of historical online conversations are between users and a first application in an online network.
14 . The method of claim 12 , further comprising:
normalizing the first set of historical online conversations.
15 . The method of claim 12 , further comprising:
translating the first set of historical online conversations.
16 . The method of claim 12 , wherein the fine-tuned conversational insights LLM is fine-tuned to generate one or more metrics regarding intent of conversations in a set of historical online conversations and to generate one or more graphical charts based on the generated one or more metrics.
17 . The method of claim 12 , wherein the first set of historical online conversations contain personal data that is not shared with the fined-tuned conversational insight LLM.
18 . A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
accessing information regarding a first conversation between a first user and a chatbot, the first conversation involving a request for analysis of a first set of historical online conversations;
generating a first prompt comprising the information regarding the first conversation, the first prompt further comprising instructions to generate instructions to retrieve and analyze historical online conversations;
submitting the first prompt to a fine-tuned conversational classification Large Language Model (LLM);
receiving, at an orchestrator, from the fine-tuned conversational classification LLM, the instructions;
retrieving, using the instructions from the fine-tuned conversational classification LLM, by the orchestrator, the first set of historical online conversations from a conversation database;
passing, by the orchestrator, the first set of historical online conversations into a fine-tuned conversational summary and translation LLM to generate summaries of the first set of historical online conversations; and
passing, by the orchestrator, the summaries of the first set of historical online conversations to a fined-tuned conversational insight LLM to generate one or more insights about the first set of historical online conversations.
19 . The non-transitory machine-readable medium of claim 18 , wherein the first set of historical online conversations are between users and a first application in an online network.
20 . The non-transitory machine-readable medium of claim 18 , wherein the operations further comprise:
normalizing the first set of historical online conversations.