SYSTEM AND METHOD FOR AGGREGATION OF DATA FROM A PLURALITY OF DATA SOURCES
System and method for interacting with a plurality of data sources are provided. A plurality of responses may be received from a plurality of data sources. The plurality of responses related to a distributed data set may be aggregated by an aggregator to generate a response to a client, the response related to a request received from the client. A plurality of data sets from which a plurality of responses is expected may be determined based on a data set distribution key that may be determined based on data included in the request from the client. Other embodiments are described and claimed.
1 . A method of providing a response to a client, the method comprising:
receiving from a plurality of data sources a respective plurality of responses related to a request received from a client and related to a data set; and
if the data set is distributed, generating a response to the client by joining the plurality of responses; and
providing the response to the client.
2 . The method of claim 1 , comprising generating a response to the client by grouping information in the plurality of responses based on a grouping clause included in the request.
3 . The method of claim 1 , comprising generating a response to the client according to a grouping operator included in the request by collecting information from a set of sorted lists according to the grouping operator, wherein the sorted lists are included in the plurality of responses.
4 . The method of claim 1 , comprising generating a response to the client by sorting information in the plurality of responses based on a sorting clause included in the request.
5 . The method of claim 1 , wherein if the request received from the client is related to an average then calculating a weighted average based on information included in the plurality of responses.
6 . The method of claim 1 , comprising:
parsing the request received from a client to determine a data set identification parameter;
designating, based on the data set identification parameter, a field included in the request as a distribution key;
determining, based on a value of the distribution key, a plurality of data sources to provide a respective plurality of responses; and
generating the response to the client based on the plurality of responses received from the plurality of data source;
7 . The method of claim 6 , comprising dynamically associating a plurality of values with the distribution key to determine a respective plurality of data sources and generating the response to the client based on responses received from the plurality of data sources.
8 . The method of claim 6 , wherein the data set identification parameter is a table name.
9 . The method of claim 6 , wherein designating a field as a distribution key is based on an association of the data set identification parameter with a field included in the request.
10 . The method of claim 6 , comprising using the value of the distribution key as an index, the index identifying a data source from which a response is expected.
11 . The method of claim 6 , comprising calculating an index indentifying a data source by providing the value of the distribution key as input to a hash function.
12 . An article comprising a non-transitory computer-readable storage medium, having stored thereon instructions, that when executed on a computer, cause the computer to:
receive from a plurality of data sources a respective plurality of responses related to a request received from a client and related to a data set; and
if the data set is distributed, generate a response to the client by joining the plurality of responses; and
provide the response to the client.
13 . The article of claim 12 , wherein the instructions when executed further result in generating a response to the client by grouping information in the plurality of responses based on a grouping clause included in the request.
14 . The article of claim 12 , wherein the instructions when executed further result in generating a response to the client according to a grouping operator included in the request by collecting information from a set of sorted lists according to the grouping operator, wherein the sorted lists are included in the plurality of responses.
15 . The article of claim 12 , wherein the instructions when executed further result in generating a response to the client by sorting information in the plurality of responses based on a sorting clause included in the request.
16 . The article of claim 12 , wherein the instructions when executed further result in if the request received from the client is related to an average then calculating a weighted average based on information included in the plurality of responses.
17 . The article of claim 12 , wherein the instructions when executed further result in:
parsing the request received from a client to determine a data set identification parameter;
designating, based on the data set identification parameter, a field included in the request as a distribution key;
determining, based on a value of the distribution key, a plurality of data sources to provide a respective plurality of responses; and
generating the response to the client based on the plurality of responses received from the plurality of data source;
18 . The article of claim 17 , wherein the instructions when executed further result in dynamically associating a plurality of values with the distribution key to determine a respective plurality of data sources and generating the response to the client based on responses received from the plurality of data sources.
19 . The article of claim 17 , wherein the data set identification parameter is a table name.
20 . The article of claim 17 , wherein designating a field as a distribution key is based on an association of the data set identification parameter with a field included in the request.
21 . The article of claim 17 , comprising using the value of the distribution key as an index, the index identifying a data source from which a response is expected.
22 . The article of claim 17 , wherein the instructions when executed further result in calculating an index indentifying a data source by providing the value of the distribution key as input to a hash function.