Combining reasoning and explicit graph walks in multi-domain queries
A data processing system implements systems and methods for performing multi-domain queries that combine explicit graph walks and fuzzy semantic reasoning of language models. These systems and methods leverage the fuzzy semantic reasoning capabilities of a language model during query execution to generate query results that are based on inferences that are not expressly defined in the graph. These techniques utilize grounding and prompt construction techniques to ensure that the language model generates an output that is consistent with a global schema that represents the types of data available from the graph data sources to significantly reduce the risk that the language model will hallucinate and generate erroneous results.
1 . A data processing system comprising:
a processor; and
a memory storing executable instructions that, when executed, cause the processor alone or in combination with other processors to perform operations of:
receiving, from a query source over a network, a query requesting information comprising one or more data elements, each data element of the one or more data elements being associated with an object type defined in a schema, the schema being associated with a plurality of data sources comprising data represented by a graph, the graph comprises nodes representing data elements and edges representing a relationship between data elements connected by the edges;
analyzing the schema to select one or more first candidate data sources comprising data relevant to the query from among the plurality of data sources;
conducting a first search of the graph associated with each of the one or more first candidate data sources for first intermediate results;
constructing a prompt to a language model based on the query and the first intermediate results to utilize fuzzy reasoning capabilities of the language model to perform a semantic analysis on the query and the first intermediate results to cause the language model to output second intermediate results;
providing the prompt as an input to the language model to cause the language model to output the second intermediate results;
analyzing the schema to select one or more second candidate data sources comprising data relevant to the query based on the second intermediate results;
conducting a second search of the graph associated with the one or more second candidate data sources to determine whether the second intermediate results are grounded by confirming that the second intermediate results are consistent with the schema and have a factual basis in data included in the graph;
responsive to the second intermediate results being grounded, conducting a third search of the graph associated with the one or more second candidate data sources to obtain third intermediate results associated with the second intermediate results to generate a response to the query; and
sending the response to the query source over the network.
2 . The data processing system of claim 1 , wherein to analyze the schema to select the one or more first candidate data sources, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
selecting the one or more first candidate data sources by minimizing a total number of data sources included in the one or more first candidate data sources, a latency associated with retrieving data from the one or more first candidate data sources, or both.
3 . The data processing system of claim 1 , wherein to conduct the second search of the graph associated with the one or more second candidate data sources to determine whether the second intermediate results are grounded, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
comparing each data element referenced in the second intermediate results with data elements referenced in the schema to determine whether each data element has a corresponding data element in the schema; and
determining that the second intermediate results are grounded in response to each of the data elements corresponding to a data element in the schema.
4 . The data processing system of claim 1 , wherein to construct the prompt to the language model, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
constructing the prompt further comprises appending the query and the first intermediate results to a prompt template, the prompt template providing instructions to the language model for a format that the language model should output the second intermediate results.
5 . The data processing system of claim 1 , wherein to analyze the schema to select one or more second candidate data sources, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
selecting the one or more second candidate data sources by minimizing a total number of data sources included in the one or more second candidate data sources, a latency associated with retrieving data from the one or more second candidate data sources, or both.
6 . The data processing system of claim 1 , wherein to conduct the third search of the graph associated with the one or more second candidate data sources to obtain data associated with the second intermediate results to generate the response to the query, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
conducting a graph walk of the one or more second candidate data sources to identify the third intermediate results associated with the second intermediate results; and
generating the response to the query based on one or more of the first intermediate results, the second intermediate results, or the third intermediate results.
7 . The data processing system of claim 1 , wherein the data processing system comprises a query execution unit, the query execution unit comprising:
a storage adapters layer, the storage adapters layer being associated with an adapter exposing each data source of the plurality of data sources as a graph;
a query planner layer analyzes the schema to identify one or more candidate data sources comprising data relevant to a respective query; and
a query runtime layer that conducts a graph walk of the graph associated with the one or more candidate data sources to retrieve search results for the respective query.
8 . A method implemented in a data processing system for providing content in response to a query, the method comprising:
receiving, from a query source over a network, the query requesting information comprising one or more data elements, each data element of the one or more data elements being associated with an object type defined in a schema, the schema being associated with a plurality of data sources comprising data represented by a graph, the graph comprises nodes representing data elements and edges representing a relationship between data elements connected by the edges;
analyzing the schema to select one or more first candidate data sources comprising data relevant to the query from among the plurality of data sources;
conducting a first search of the graph associated with each of the one or more first candidate data sources for first intermediate results;
constructing a prompt to a language model based on the query and the first intermediate results to utilize fuzzy reasoning capabilities of the language model to perform a semantic analysis on the query and the first intermediate results to cause the language model to output second intermediate results;
providing the prompt as an input to the language model to cause the language model to output the second intermediate results;
analyzing the schema to select one or more second candidate data sources comprising data relevant to the query based on the second intermediate results;
conducting a second search of the graph associated with the one or more second candidate data sources to determine whether the second intermediate results are grounded by confirming that the second intermediate results are consistent with the schema and have a factual basis in data included in the graph;
responsive to the second intermediate results being grounded, conducting a third search of the graph associated with the one or more second candidate data sources to obtain third intermediate results associated with the second intermediate results to generate a response to the query; and
sending the response to the query source over the network.
9 . The method of claim 8 , wherein analyzing the schema to select the one or more first candidate data sources further comprises:
selecting the one or more first candidate data sources by minimizing a total number of data sources included in the one or more first candidate data sources, a latency associated with retrieving data from the one or more first candidate data sources, or both.
10 . The method of claim 8 , wherein conducting the second search of the graph associated with the one or more second candidate data sources to determine whether the second intermediate results are grounded further comprises:
comparing each data element referenced in the second intermediate results with data elements referenced in the schema to determine whether each data element has a corresponding data element in the schema; and
determining that the second intermediate results are grounded in response to each of the data elements corresponding to a data element in the schema.
11 . The method of claim 8 , wherein constructing the prompt to the language model further comprises:
constructing the prompt further comprises appending the query and the first intermediate results to a prompt template, the prompt template providing instructions to the language model for a format that the language model should output the second intermediate results.
12 . The method of claim 8 , wherein analyzing the schema to select one or more second candidate data sources further comprises:
selecting the one or more second candidate data sources by minimizing a total number of data sources included in the one or more second candidate data sources, a latency associated with retrieving data from the one or more second candidate data sources, or both.
13 . The method of claim 8 , wherein conducting the third search of the graph associated with the one or more second candidate data sources to obtain data associated with the second intermediate results to generate the response to the query further comprising:
conducting a graph walk of the one or more second candidate data sources to identify the third intermediate results associated with the second intermediate results; and
generating the response to the query based on one or more of the first intermediate results, the second intermediate results, or the third intermediate results.
14 . The method of claim 8 , wherein the data processing system comprises a query execution unit, the method further comprising:
exposing each data source of the plurality of data sources as a graph with an adapter of a storage adapters layer of the query execution unit;
analyzing the schema using a query planner layer of the query execution unit to identify one or more candidate data sources comprising data relevant to a respective query; and
conducting, using a query runtime layer of the query execution unit, a graph walk of the graph associated with the one or more candidate data sources to retrieve search results for the respective query.
15 . A data processing system comprising:
a processor; and
a memory storing executable instructions that, when executed, cause the processor alone or in combination with other processors to perform operations of:
receiving, from a query source over a network, a natural language prompt requesting information from one or more data sources;
analyzing the natural language prompt to identify one or more data elements referenced in the natural language prompt, each data element of the one or more data elements being associated with an object type defined in a schema, the schema being associated with a plurality of data sources comprising data represented by a graph, the graph comprises nodes representing data elements and edges representing a relationship between data elements connected by the edges;
constructing a query based on the natural language prompt and the one or more data elements referenced in the natural language prompt;
analyzing the schema to select one or more first candidate data sources comprising data relevant to the query from among the plurality of data sources;
conducting a first search of the graph associated with each of the one or more first candidate data sources for first intermediate results;
constructing a prompt to a language model based on the query and the first intermediate results to perform a semantic analysis on the query and the first intermediate results to cause the language model to output second intermediate results;
providing the prompt as an input to the language model to cause the language model to output the second intermediate results;
analyzing the schema to select one or more second candidate data sources comprising data relevant to the query based on the second intermediate results;
conducting a second search of the graph associated with the one or more second candidate data sources to determine whether the second intermediate results are grounded by confirming that the second intermediate results are consistent with the schema and have a factual basis in data included in the graph;
responsive to the second intermediate results being grounded, conducting a third search of the graph associated with the one or more second candidate data sources to obtain third intermediate results associated with the second intermediate results to generate a response to the query; and
sending the response to the query source over the network.
16 . The data processing system of claim 15 , wherein to analyze the schema to select the one or more first candidate data sources, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
selecting the one or more first candidate data sources by minimizing a total number of data sources included in the one or more first candidate data sources, a latency associated with retrieving data from the one or more first candidate data sources, or both.
17 . The data processing system of claim 15 , wherein to conduct the second search of the graph associated with the one or more second candidate data sources to determine whether the second intermediate results are grounded, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
comparing each data element referenced in the second intermediate results with data elements referenced in the schema to determine whether each data element has a corresponding data element in the schema; and
determining that the second intermediate results are grounded in response to each of the data elements corresponding to a data element in the schema.
18 . The data processing system of claim 15 , wherein to construct the prompt to the language model, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
constructing the prompt further comprises appending the query and the first intermediate results to a prompt template, the prompt template providing instructions to the language model for a format that the language model should output the second intermediate results.
19 . The data processing system of claim 15 , wherein to analyze the schema to select one or more second candidate data sources, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
selecting the one or more second candidate data sources by minimizing a total number of data sources included in the one or more second candidate data sources, a latency associated with retrieving data from the one or more second candidate data sources, or both.
20 . The data processing system of claim 15 , wherein to conduct the third search of the graph associated with the one or more second candidate data sources to obtain data associated with the second intermediate results to generate the response to the query, the memory further includes instructions configured to cause the processor alone or in combination with other processors to perform operations of:
conducting a graph walk of the one or more second candidate data sources to identify the third intermediate results associated with the second intermediate results; and
generating the response to the query based on one or more of the first intermediate results, the second intermediate results, or the third intermediate results.