Generating a response for a communication session based on previous conversation content using a large language model
An example operation may include one or more of receiving interaction content from a communication session between a source device and a service provider device of a service provider, identifying a search criteria from the interaction content, retrieving a subset of vectors from a plurality of vectors stored in a vector database based on the search criteria of the interaction content, wherein the subset of vectors includes previous interaction content with the service provider, generating a response for the communication session based on execution of a large language model (LLM) on the subset of vectors, and outputting the response to at least one of the source device and the service provider device during the communication session.
1 . An apparatus comprising:
a memory; and
a processor coupled to the memory, the processor configured to:
train an artificial intelligence (AI) model to identify contextual attributes from communication content and embed the contextual attributes into vectors;
receive interaction content from a communication session between a first device and a second device,
identify contextual attributes from the communication session based on execution of the AI model on the interaction content and embed the contextual attributes into metadata of a vector,
identify at least one vector stored in a vector database that includes corresponding metadata that contains the contextual attributes,
generate a response based on execution of a large language model (LLM) on the at least one vector, and
output the response to the first device during the communication session.
2 . The apparatus of claim 1 , wherein the processor is further configured to generate a prompt which includes the at least one vector and input the prompt to the LLM during the execution of the LLM.
3 . The apparatus of claim 1 , wherein the processor is further configured to embed the interaction content into the vector and identify the at least one vector based on a similarity between the vector and a plurality of vectors in vector space.
4 . The apparatus of claim 1 , wherein the processor is configured to identify a tone and a speech rate of the communication session based on execution of an additional LLM on the interaction content, and embed the tone and the speech rate into the at least one vector.
5 . The apparatus of claim 1 , wherein the AI model comprises a multi-headed attention mechanism corresponding to a plurality of contextual attributes and the processor is configured to identify a plurality of values for the plurality of contextual attributes from the interaction content based on the multi-headed attention mechanism.
6 . The apparatus of claim 1 , wherein the processor is configured to simultaneously perform the communication session and execute the LLM based on real-time interaction content from the communication session.
7 . A method comprising:
training an artificial intelligence (AI) model to identify contextual attributes from communication content and embed the contextual attributes into vectors;
receiving interaction content from a communication session between a first device and a second device;
identifying contextual attributes from the communication session based on execution of the AI model on the interaction content and embedding the contextual attributes into metadata of a vector;
identifying at least one vector stored in a vector database that includes corresponding metadata that contains the contextual attributes;
generating a response based on execution of a large language model (LLM) on the at least one vector; and
outputting the response to the first device during the communication session.
8 . The method of claim 7 , wherein the method further comprises generating a prompt which includes the at least one vector and the generating comprises inputting the prompt to the LLM during the execution of the LLM.
9 . The method of claim 8 , wherein the method further comprises generating additional text and adding the additional text to the prompt prior to inputting the prompt to the LLM during the execution of the LLM.
10 . The method of claim 7 , wherein the method further comprises embedding the interaction content into the vector and identifying the at least one vector based on a similarity between the vector and a plurality of vectors in vector space.
11 . The method of claim 7 , wherein the identifying comprises identifying a tone and a speech rate of the communication session based on execution of an additional LLM on the interaction content, and embedding the tone and the speech rate into the at least one vector.
12 . The method of claim 7 , wherein the AI model comprises a multi-headed attention mechanism corresponding to a plurality of contextual attributes and the identifying comprises identifying a plurality of values for the plurality of contextual attributes from the interaction content based on the multi-headed attention mechanism.
13 . The method of claim 7 , wherein the generating the response comprises simultaneously performing the communication session and executing the LLM based on real-time interaction content from the communication session.
14 . A computer-readable storage medium comprising instructions stored therein which when executed by a processor cause the processor to perform:
training an artificial intelligence (AI) model to identify contextual attributes from communication content and embed the contextual attributes into vectors;
receiving interaction content from a communication session between a first device and a second device;
identifying contextual attributes from the communication session based on execution of the AI model on the interaction content and embedding the contextual attributes into metadata of a vector;
identifying at least one vector from among a plurality of vectors stored in a vector database which includes corresponding metadata that contains the contextual attributes;
generating a response based on execution of a large language model (LLM) on the at least one vector; and
outputting the response to the first device during the communication session.
15 . The computer-readable storage medium of claim 14 , wherein the processor is further configured to perform generating a prompt which includes the at least one vector and the generating comprises inputting the prompt to the LLM during the execution of the LLM.
16 . The computer-readable storage medium of claim 15 , wherein the processor is further configured to perform generating additional text and adding the additional text to the prompt prior to inputting the prompt to the LLM during the execution of the LLM.
17 . The computer-readable storage medium of claim 14 , wherein the processor is further configured to perform embedding the interaction content into a search vector and identifying the at least one vector based on a similarity between the search vector and the plurality of vectors in vector space.
18 . The computer-readable storage medium of claim 14 , wherein the identifying comprises identifying a tone and a speech rate of the communication session based on execution of an additional LLM on the interaction content, and embedding the tone and the speech rate into the at least one vector.
19 . The computer-readable storage medium of claim 14 , wherein the AI model comprises a multi-headed attention mechanism corresponding to a plurality of contextual attributes and the identifying comprises identifying a plurality of values for the plurality of contextual attributes from the interaction content based on the multi-headed attention mechanism.
20 . The apparatus of claim 1 , wherein the processor is configured to identify a cluster of vectors stored within the vector database that include a shared contextual label identifying the contextual attributes, and generate the response based on the cluster of vectors.