Database systems and methods for automated conversational responses
View Patent ↗Database systems and methods are provided for managing usage of large language models (LLMs). One method involves determining a numerical representation of a conversational input to a user interface, identifying a semantically similar subset of prior conversational inputs based at least in part on the numerical representation of the conversational input, and determining numerical representations of respective conversational responses generated by a language model responsive to the respective prior conversational input of the semantically similar subset. When the numerical representations of the respective conversational responses satisfy a semantic similarity threshold, the method automatically generates an automated response to the conversational input based at least in part on one or more prior conversational responses and automatically provides the automated response to the user interface responsive to the conversational input.
1 . A method comprising:
determining a numerical representation of a conversational input to a user interface;
identifying, based at least in part on the numerical representation of the conversational input, a semantically similar subset of prior conversational inputs previously sent to a large language model-based (LLM-based) service from a plurality of prior conversational inputs having associated conversational responses maintained at a database system, wherein a respective prior conversational input of the semantically similar subset has a respective conversational response generated by the LLM-based service responsive to the respective prior conversational input associated therewith, resulting an associated subset of prior conversational responses provided by the LLM-based service responsive to the semantically similar subset of prior conversational inputs;
determining numerical representations of the respective conversational responses of the associated subset of prior conversational responses provided by the LLM-based service; and
when the numerical representations of the respective conversational responses satisfy a semantic similarity threshold for homogeneity across the prior conversational responses provided by the LLM-based service:
automatically generating a synthetic LLM response to the conversational input based at least in part on one or more prior conversational responses of the associated subset of prior conversational responses provided by the LLM-based service responsive to the semantically similar subset of prior conversational inputs; and
automatically providing the synthetic LLM response to the user interface responsive to the conversational input.
2 . The method of claim 1 , further comprising removing personal identifying information from the conversational input prior to determining the numerical representation of the conversational input.
3 . The method of claim 2 , further comprising supplementing the synthetic LLM response with the personal identifying information prior to automatically the automated synthetic LLM response to the user interface.
4 . The method of claim 1 , wherein identifying the semantically similar subset of prior conversational inputs comprises identifying a cluster group of prior input prompts previously sent to the LLM-based service having respective numerical representations semantically similar to the numerical representation of the conversational input.
5 . The method of claim 1 , further comprising filtering the semantically similar subset of prior conversational inputs based at least in part on contextual information associated with the conversational input prior to identifying the associated subset of prior conversational responses.
6 . The method of claim 5 , wherein the contextual information includes at least one of an identifier associated with a user providing the conversational input and a database object type associated with the conversational input.
7 . At least one non-transitory machine-readable storage medium that provides instructions that, when executed by at least one processor, are configurable to cause the at least one processor to perform operations comprising:
determining a numerical representation of a conversational input to a user interface;
identifying, based at least in part on the numerical representation of the conversational input, a semantically similar subset of prior conversational inputs previously sent to a large language model-based (LLM-based) service from a plurality of prior conversational inputs having associated conversational responses maintained at a database system, wherein a respective prior conversational input of the semantically similar subset has a respective conversational response generated by the LLM-based service responsive to the respective prior conversational input associated therewith, resulting an associated subset of prior conversational responses provided by the LLM-based service responsive to the semantically similar subset of prior conversational inputs;
determining numerical representations of the respective conversational responses of the associated subset of prior conversational responses provided by the LLM-based service; and
when the numerical representations of the respective conversational responses satisfy a semantic similarity threshold for homogeneity across the prior conversational responses provided by the LLM-based service:
automatically generating a synthetic LLM response to the conversational input based at least in part on one or more prior conversational responses of the associated subset of prior conversational responses provided by the LLM-based service responsive to the semantically similar subset of prior conversational inputs; and
automatically providing the synthetic LLM response to the user interface responsive to the conversational input.
8 . The at least one non-transitory machine-readable storage medium of claim 7 , wherein the instructions are configurable to cause the at least one processor to remove personal identifying information from the conversational input prior to determining the numerical representation of the conversational input.
9 . The at least one non-transitory machine-readable storage medium of claim 8 , wherein the instructions are configurable to cause the at least one processor to supplement the synthetic LLM response with the personal identifying information prior to automatically providing the synthetic LLM response to the user interface.
10 . The at least one non-transitory machine-readable storage medium of claim 7 , wherein:
the semantically similar subset of prior conversational inputs comprises a subset of prior input prompts previously sent to the LLM-based service; and
the associated subset of prior conversational responses comprises conversational responses provided by the LLM-based service in response to receiving respective ones of the subset of prior input prompts.
11 . The at least one non-transitory machine-readable storage medium of claim 7 , wherein the instructions are configurable to cause the at least one processor to identify a cluster group of prior input prompts previously sent to the LLM-based service having respective numerical representations semantically similar to the numerical representation of the conversational input.
12 . The at least one non-transitory machine-readable storage medium of claim 7 , wherein the instructions are configurable to cause the at least one processor to filter the semantically similar subset of prior conversational inputs based at least in part on contextual information associated with the conversational input prior to identifying the associated subset of prior conversational responses.
13 . The at least one non-transitory machine-readable storage medium of claim 12 , wherein the contextual information includes at least one of an identifier associated with a user providing the conversational input and a database object type associated with the conversational input.
14 . A computing system comprising:
at least one non-transitory machine-readable storage medium that stores software; and
at least one processor, coupled to the at least one non-transitory machine-readable storage medium, to execute the software that implements a large language model (LLM) management service and that is configurable to perform operations comprising:
determining a numerical representation of a conversational input to a user interface;
identifying, based at least in part on the numerical representation of the conversational input, a semantically similar subset of prior conversational inputs previously sent to an LLM-based service from a plurality of prior conversational inputs having associated conversational responses maintained at a database system, wherein a respective prior conversational input of the semantically similar subset has a respective conversational response generated by the LLM-based service responsive to the respective prior conversational input associated therewith, resulting an associated subset of prior conversational responses provided by the LLM-based service responsive to the semantically similar subset of prior conversational inputs;
determining numerical representations of the respective conversational responses of the associated subset of prior conversational responses provided by the LLM-based service; and
when the numerical representations of the respective conversational responses satisfy a semantic similarity threshold for homogeneity across the prior conversational responses provided by the LLM-based service:
automatically generating a synthetic LLM response to the conversational input based at least in part on one or more prior conversational responses of the associated subset of prior conversational responses provided by the LLM-based service responsive to the semantically similar subset of prior conversational inputs; and
automatically providing the synthetic LLM response to the user interface responsive to the conversational input.
15 . The computing system of claim 14 , wherein the LLM management service is configurable to:
remove personal identifying information from the conversational input prior to determining the numerical representation of the conversational input; and
supplement the synthetic LLM response with the personal identifying information prior to automatically providing the synthetic LLM response to the user interface.
16 . The computing system of claim 14 , wherein the LLM management service is configurable to filter the semantically similar subset of prior conversational inputs based at least in part on contextual information associated with the conversational input prior to identifying the conversational responses provided by the LLM-based service in response to receiving respective ones of a filtered semantically similar subset of prior conversational inputs.
17 . The method of claim 1 , wherein:
the semantically similar subset of prior conversational inputs comprises a subset of prior input prompts previously sent to the LLM-based service;
each prior input prompt of the subset of prior input prompts has a respective numerical representation within a threshold similarity of the numerical representation of the conversational input;
each prior input prompt of the subset of prior input prompts has a respective conversational response by the LLM-based service in response to receiving the respective prior input prompts; and
the associated subset of prior conversational responses comprises the respective conversational responses provided by the LLM-based service in response to receiving respective ones of the subset of prior input prompts.
18 . The method of claim 1 , wherein:
the conversational input and the prior conversational inputs are unstructured and free form using natural language; and
numerical representations of respective prior conversational inputs of the semantically similar subset of prior conversational inputs are within a threshold degree of similarity of the numerical representation of the conversational input.
19 . The method of claim 1 , further comprising filtering the prior conversational inputs previously sent to the LLM-based service based at least in part on a database object type associated with the conversational input to obtain the semantically similar subset of prior conversational inputs by removing a second subset of the prior conversational inputs relating to a different database object type than the database object type.
20 . The method of claim 1 , further comprising filtering the prior conversational inputs previously sent to the LLM-based service based at least in part on an identifier associated with a user providing the conversational input to remove a second subset of the prior conversational inputs and obtain the semantically similar subset of prior conversational inputs associated with a same tenant, organization or user as the conversational input.