Dynamic construction of large language model prompts
Example solutions for reducing the likelihood of hallucinations by language models, such as large language models (LLMs) are disclosed. By injecting a sufficient range and quantity of curated factual data into a prompt, the likelihood of a hallucination by an LLM may be reduced. This enables language models to be used in a wider range of settings, in which fabrication of facts is problematic, while reducing the need for a human to carefully check the generated text for accuracy. Examples include: generating a summary of a transcript using a summarization model; extracting topic-specific data from stored data using a scoring model; dynamically generating a language model prompt using the topic-specific data and the summary; and generating an output text using a language model and the language model prompt.
1 . A system comprising:
a processor; and
a computer-readable medium storing instructions that are operative upon execution by the processor to:
receiving a transcript of a conversation between participants;
generate a summary of the transcript using a summarization model, the summary relating to a topic of the conversation;
extract topic-specific data associated with the topic from a customer relations management (CRM) or enterprise suite data set using a scoring model, the CRM or enterprise data set excluding the transcript;
dynamically generate a large language model (LLM) prompt based on the summary by resolving a difference between the topic-specific data and the summary in favor of the topic-specific data; and
generate, by an LLM model, an output text based on the LLM prompt.
2 . The system of claim 1 , wherein the instructions are further operative to:
transmit, across a computer network, the output text as an electronic mail (email) message.
3 . The system of claim 1 , wherein the topic of the conversation comprises an action item extracted from the transcript.
4 . The system of claim 1 , wherein the transcript is generated from audio data using automatic speech recognition (ASR).
5 . The system of claim 1 , wherein the LLM prompt is further generated based on a count of speakers identified in the summary.
6 . The system of claim 1 , wherein the topic-specific data comprises calendar information from the CRM or enterprise suite data set.
7 . The system of claim 1 , wherein the topic-specific data comprises a CRM interaction history of one of the participants.
8 . A computer-implemented method comprising:
receiving a transcript of a conversation between participants;
generating a summary of the transcript of using a summarization model, the summary relating to a topic of the conversation;
extracting topic-specific data associated with the topic of conversation from a customer relations management (CRM) or enterprise suite data set using a scoring model, the CRM or enterprise data set excluding the transcript;
dynamically generating a large language model (LLM) prompt based on the summary by resolving a difference between the topic-specific data and the summary in favor of the topic-specific data; and
generating, by an LLM model, an output text based on the LLM prompt.
9 . The computer-implemented method of claim 8 , further comprising:
transmitting, across a computer network, the output text as an electronic mail (email) message.
10 . The computer-implemented method of claim 8 , wherein the topic of the conversation comprises an action item extracted from the transcript.
11 . The computer-implemented method of claim 8 , wherein the transcript is generated from audio data using automatic speech recognition (ASR).
12 . The computer-implemented method of claim 8 , wherein the LLM prompt is further generated based on a count of speakers identified in the summary or an interaction history associated with a name identified in the summary.
13 . The computer-implemented method of claim 8 , wherein the topic-specific data comprises calendar information from the CRM or enterprise suite data set.
14 . The computer-implemented method of claim 8 , wherein the topic-specific data comprises a CRM interaction history of one of the participants.
15 . A computer storage device having computer-executable instructions stored thereon, which, on execution by one or more processors of a system, cause the system to perform operations comprising:
receiving a transcript of a conversation between participants;
generating a summary of the transcript using a summarization model, the summary relating to a topic of the conversation;
extracting topic-specific data associated with the topic from a customer relations management (CRM) or enterprise suite data set using a scoring model, the CRM or enterprise data set excluding the transcript;
dynamically generating a large language model (LLM) prompt based on the summary by resolving a difference between the topic-specific data and the summary in favor of the topic-specific data; and
generating, via an LLM model, an output text based on the LLM prompt.
16 . The computer storage device of claim 15 , wherein the operations further comprise:
transmitting, across a computer network, the output text as an electronic mail (email) message.
17 . The computer storage device of claim 15 , wherein the topic of the conversation comprises an action item extracted from the transcript.
18 . The computer storage device of claim 15 , wherein the transcript is generated from audio data using automatic speech recognition (ASR).
19 . The computer storage device of claim 15 , wherein the LLM prompt is further generated based on a count of speakers identified in the summary or an interaction history associated with a name identified in the summary.
20 . The computer storage device of claim 15 , wherein the topic-specific data comprises a CRM interaction history of one of the participants.