Systems and methods for query evaluation over distributed linked data stores
A method for query evaluation comprises receiving a query over a set of distributed data sources, decomposing the query into a set of sub-queries of the query, evaluating each sub-query in the set of sub-queries with respect to each data source in the set of distributed data sources, wherein evaluating comprises determining which data sources in the set of distributed data sources are capable of answering each sub-query and at what cost, computing a set of distributed plans by composing one or more of the sub-queries in one or more of the data sources, evaluating each plan in the set of distributed plans, selecting a sub-set of plans from the set of distributed plans to be executed for responding to the query, executing the selected sub-set of plans, and returning results of the query.
1. A method for query evaluation, comprising:
implementing a data query evaluation system at each data source of a set of distributed and networked linked data sources, wherein the distributed and network linked data sources respectively correspond to a plurality of nodes connected over a network, wherein the data query evaluation system at each data source is configured for:
receiving a query over the set of distributed and network linked data sources;
decomposing the query into a set of sub-queries of the query;
evaluating each sub-query in the set of sub-queries with respect to each data source in the set of distributed and network linked data sources, wherein evaluating comprises determining which data sources in the set of distributed and network linked data sources are capable of answering each sub-query, wherein each data source of the set of distributed and networked linked data sources dynamically determines whether it is capable of answering each sub-query;
summarizing results of the evaluating of each sub-query;
exchanging between the distributed and networked linked data sources the summarized results of the evaluating, wherein each data source of the set of distributed and networked linked data sources dynamically provides the summarized results;
executing, in parallel by the distributed and networked linked data sources, a distributed query evaluation process to compute a plurality of combinations of query results from the distributed and networked linked data sources which provide an answer to the query;
computing a set of distributed plans by composing one or more of the sub-queries in one or more of the distributed and networked linked data sources;
evaluating each plan in the set of distributed plans;
selecting a sub-set of plans from the set of distributed plans to be executed for responding to the query;
executing the selected sub-set of plans, wherein executing the selected sub-set of plans comprises:
an exchange of data between each of the distributed and networked linked data sources that are part of the selected sub-set of plans;
executing a plurality of sub-queries in respective ones of the distributed and networked linked data sources that are part of the selected sub-set of plans; and
forwarding results of the execution of one or more given sub-queries of the plurality sub-queries to one or more data sources of the distributed and networked linked data sources that are part of the selected sub-set of plans, and which did not perform the one or more given sub-queries, wherein the results comprise answers and partial answers to the given sub-queries; and
returning results of the query;
wherein the implementing of the data query evaluation system at each data source is performed without: (i) transmission of queried data to a central repository; (ii) using a central federator for coordinating and merging the partial answers from the one or more data sources of the distributed and networked linked data sources that are part of the selected sub-set of plans; and (iii) using a global schema to evaluate the query; and
wherein the steps of the method are performed via at least one processing device and at least one memory.
2. The method according to claim 1 , wherein the query comprises a graph query and the data sources comprise graph data sources.
3. The method according to claim 1 , wherein the summarized results include an identification of local statistics and remote statistics for each data source.
4. The method according to claim 1 , further comprising determining a nature of information being provided by a data source in response to a composed sub-query for that data source, and determining a cost associated with obtaining the information.
5. The method according to claim 4 , wherein the cost is based on at least one of memory and time needed for the composed sub-query.
6. The method according to claim 1 , wherein evaluating each plan comprises comparing the set of distributed plans with each other with respect to a cost of each plan.
7. The method according to claim 1 , wherein the one or more of the sub-queries are composed in each data source determined to be capable of answering a sub-query.
8. The method according to claim 1 , further comprising compiling results of the executing of the selected sub-set of plans.