Granted Patent B2

US 12,561,351 · App. 18/473,989 · Granted Feb 24, 2026

Inventors: Santosh Addanki (Cupertino, CA); Soujanya Lanka (Singapore, SG); Nandana Murthy (San Jose, CA); Koteswara Rao Pathuri (Georgetown, TX); Bineet Ranjan (Bengaluru, IN); Liang Xi (Elkhorn, NE); Xiaoying Han (Santa Clara, CA); Raghotham Sripadraj (Bengaluru, IN)

Assignee: PAYPAL, INC.

G06F16/3329G06F16/3326G06F16/36

Abstract

Claims (66)

1 . A system, comprising:

a non-transitory memory; and

one or more hardware processors coupled with the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:

receiving, from a user device, a user query;

generating first embeddings based on parsing a plurality of words of the user query, wherein the first embeddings represent a contextual meaning of the user query;

determining whether a response to the user query is stored in a cache storage based on comparing the first embeddings against a plurality of cache keys associated with the cache storage;

in response to determining that the response to the user query is not stored in the cache storage, querying a corpus of documents using a text-based search model based on one or more keywords extracted from the user query, wherein a first set of documents from the corpus of documents is obtained based on the querying the corpus of documents using the text-based search model;

querying the corpus of documents using a semantic search model based on the first embeddings, wherein a second set of documents from the corpus of documents is obtained based on the querying the corpus of documents using the semantic search model;

selecting, from the first set of documents and the second set of documents, one or more documents having relevancy scores associated with the user query above a threshold;

generating a prompt for a generative artificial intelligence (AI) model based on combining the user query and the one or more documents;

generating, using the generative AI model and based on the prompt, a response to the user query, wherein the response is generated further based on information from the one or more documents;

providing the response to the user device;

obtaining benchmark embeddings representing a benchmark response associated with the user query;

generating second embeddings representing the response generated using the generative AI model; and

modifying the corpus of documents based on a difference between the second embeddings and the benchmark embeddings.

2 . The system of claim 1 , wherein the user query is in a natural language format.

3 . The system of claim 1 , wherein the text-based search model is configured to identify, from the corpus of documents, the first set of documents based on the one or more keywords extracted from the user query.

4 . The system of claim 1 , wherein the semantic-based search model is configured to identify, from the corpus of documents, the second set of documents based on the contextual meaning of the user query.

5 . The system of claim 1 , wherein the first embeddings are generated further based on parsing the plurality of words forward and backward.

6 . The system of claim 1 , wherein the operations further comprise:

analyzing the first set of documents and the second set of documents; and

ranking the first set of documents and the second set of documents based on the analyzing, wherein the one or more documents are selected based on the ranking.

7 . The system of claim 1 , wherein the modifying the corpus of documents comprises at least one of adding an additional document to the corpus of documents or removing a document from the corpus of documents.

8 . A method, comprising:

receiving, by a computer system and from a user device, a user query;

generating, by the computer system, first embeddings representing the user query;

accessing, from a plurality of corpuses of documents, a corpus of documents based on a particular domain associated with the user query;

querying, by the computer system, the corpus of documents using a text-based search model based on one or more keywords extracted from the user query, wherein a first set of documents is obtained based on the querying the corpus of documents using the text-based search model;

querying, by the computer system, the corpus of documents using a semantic search model based on the first embeddings, wherein a second set of documents is obtained based on the querying the corpus of documents using the semantic search model;

determining, by the computer system and from the first set of documents and the second set of documents, that two or more documents are associated with a response to the user query;

generating, using a machine learning model and based on an input comprising the user query and the two or more documents, the response to the user query, wherein the response comprises content generated based on information extracted from at least two documents in the two or more documents;

generating second embeddings representing the response generated using the machine learning model;

accessing benchmark embeddings representing a benchmark response associated with the user query; and

modifying the corpus of documents based on comparing the second embeddings against the benchmark embeddings.

9 . The method of claim 8 , wherein the user query is a first user query that is part of a dialogue between a user of the user device and a knowledge bot, and wherein the method further comprises:

retrieving a chat history associated with the user of the user device, wherein the chat history comprises one or more user queries previously submitted by the user and one or more responses generated by the machine learning model for the one or more user queries;

deriving a context based on the chat history; and

modifying the first user query based on the context, wherein the two or more documents are determined to be associated with the first user query based on the modified first user query.

10 . The method of claim 9 , wherein the modifying the first user query comprises at least one of adding one or more words to, deleting one or more words from, or revising one or more words from the first user query based on at least one of the one or more user queries or the one or more responses.

11 . The method of claim 9 , wherein the input further comprises the context and the modified first user query.

12 . The method of claim 8 , further comprising:

determining, from a plurality of domains, that the user query is associated with the particular domain based on analyzing the user query.

13 . The method of claim 8 , further comprising:

determining a score for each document in the first set of documents and the second set of documents;

ranking the first set of documents and second set of documents based on the score determined for each document in the first set of documents and the second set of documents; and

selecting, from the first set of documents and the second set of documents, the two or more documents based on the ranking.

14 . The method of claim 8 , wherein the response comprises words in a natural language format.

15 . A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:

generating, based on a query, first embeddings that represent a semantic meaning of the query;

querying a corpus of documents using a text-based search model based on one or more keywords extracted from the query, wherein a first set of documents is obtained based on the querying the corpus of documents using the text-based search model;

querying the corpus of documents using a semantic search model based on the first embeddings, wherein a second set of documents is obtained based on the querying the corpus of documents using the semantic search model;

determining, from the first set of documents and the second set of documents, that two or more documents are associated with a response to the query;

generating, using a machine learning model and based on a prompt that includes the query and the two or more documents, the response to the query, wherein the response comprises content generated based on information extracted from at least two documents in the two or more documents;

obtaining benchmark embeddings representing a benchmark response associated with the query;

generating second embeddings representing the response generated using the machine learning model; and

modifying the corpus of documents based on a difference between the second embeddings and the benchmark embeddings.

16 . The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise:

comparing the first embeddings against a plurality of cache keys of a cache storage; and

determining that the query is not related to an entry in the cache storage based on the comparing.

17 . The non-transitory machine-readable medium of claim 15 , wherein the semantic-based search model is configured to (i) derive a contextual meaning of the query and (ii) identify, from the corpus documents, the second set of documents based on the contextual meaning of the query.

18 . The non-transitory machine-readable medium of claim 15 , wherein the first embeddings are generated based on parsing a plurality of words of the query forward and backward.

19 . The non-transitory machine-readable medium of claim 15 , wherein the query is a first query that is part of a dialogue between a user of a user device and a knowledge bot during a session between the user and a service provider, and wherein the operations further comprise:

retrieving a chat history associated with the user of the user device, wherein the chat history comprises one or more queries previously submitted by the user and one or more responses generated by the machine learning model for the one or more queries;

deriving a context based on the chat history; and

modifying the first query based on the context, wherein the two or more documents are determined to be associated with the response to the first query based on the modified first query.

20 . The non-transitory machine-readable medium of claim 19 , wherein the modifying the first query comprises at least one of adding one or more words to, deleting one or more words from, or revising one or more words from the first query based on at least one of the one or more queries or the one or more responses.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (43)