SYSTEMS AND METHODS FOR MERGING PARTIALLY AGGREGATED QUERY RESULTS
Systems and methods for merging partially aggregated query results are provided. A partially aggregated query result is determined. Each query of a plurality of queries is executed on a plurality of events at a defined schedule and a time duration. A key and a value of the partially aggregated query result are identified. It is determined whether a function for the partially aggregated query result is identified. If so, a related partially aggregated query result is determined using the key. The partially aggregated query result is merged with the related partially aggregated query result.
1 . A method for processing aggregated query results, the method comprising:
determining a partially aggregated query result, wherein each query of a plurality of queries is executed on a plurality of events at a defined schedule and a time duration;
identifying a key and a value of the partially aggregated query result;
determining whether a function for the partially aggregated query result is identified;
determining a related partially aggregated query result of a plurality of partially aggregated query results using the key; and
merging, at a local memory of a computing device, the partially aggregated query result and the related partially aggregated query result.
2 . The method of claim 1 , wherein merging comprises:
applying the function to the value of the partially aggregated query result and the value of the related partially aggregated query result.
3 . The method of claim 1 , further comprising:
storing a complete aggregation of the query result in a persistent storage, wherein the complete aggregation of the query result is determined upon merging the partially aggregated query result and the related partially aggregated query result.
4 . The method of claim 1 , wherein the partially aggregated query result is generated by a distributed manager of a network system, and the partially aggregated query result is received by a local manager of the network system.
5 . The method of claim 1 , further comprising:
detecting a query for real-time data;
issuing the query for real-time data on a persistent storage, wherein the persistent storage includes the plurality of partially aggregated query result;
determining a result of issuing the query on the persistent storage; and
determining an in-memory aggregation of the query for real-time data, wherein the complete aggregation of the query result is generated using the result of issuing the query on the persistent storage and the in-memory aggregation.
6 . The method of claim 1 , further comprising:
receiving, at a local memory of the computing device, a plurality of events in an event stream;
determining the plurality of events are out-of-order events;
determining a query result for each of the plurality of events; and
determining a partially aggregated query result based on the query result for each of the plurality of events.
7 . A system for processing partially aggregated query results, the system comprising:
a persistent store for storage of partially aggregated query results and complete query results; and
a computer that includes:
a trend aggregation module; and
a memory for merging of partially aggregated query results;
wherein the trend aggregation module is configured to:
determine a partially aggregated query result, wherein each query of a plurality of queries is executed on a plurality of events at a defined schedule and a time duration;
identify a key and a value of the partially aggregated query result;
determine whether a function for the partially aggregated query result is identified;
determine a related partially aggregated query result of a plurality of partially aggregated query results using the key; and
merge the partially aggregated query result and the related partially aggregated query result.
8 . The system of claim 7 , wherein merging comprises:
applying the function to the value of the partially aggregated query result and to the value of the related partially aggregated query result.
9 . The system of claim 7 , wherein the trend aggregation module is further configured to:
store a complete query result in the persistent storage, wherein the complete query result is determined upon the partially aggregated query result and the related partially aggregated query result.
10 . The system of claim 7 , wherein the trend aggregation module is further configured to:
detect a query for real-time data;
issue the query for real-time data on the persistent storage;
determine a result of issuing the query on the persistent storage; and
determine an in-memory aggregation of the query for real-time data, wherein a complete aggregation of the query result is generated using the result of issuing the query on the persistent storage and the in-memory aggregation.
11 . The system of claim 7 , wherein the memory is further configured to receive a plurality of events in an event stream, and wherein the trend aggregation module is further configured to:
determine the plurality of events are out-of-order events;
determine a query result for each of the plurality of events; and
determine a partially aggregated query result based on the query result for each of the plurality of events.
12 . A non-transitory computer-readable medium storing a plurality of instructions to control a data processor to process partially aggregated query results, the plurality of instructions comprising instructions that cause the data processor to:
determine a partially aggregated query result, wherein each query of a plurality of queries is executed on a plurality of events at a defined schedule and a time duration;
identify a key and a value of the partially aggregated query result;
determine whether a function for the partially aggregated query result is identified;
determine a related partially aggregated query result of a plurality of partially aggregated query results using the key; and
merge, at a local memory of a computing device, the partially aggregated query result and the related partially aggregated query result.
13 . The non-transitory computer-readable medium of claim 12 , wherein the instructions that cause the data processor to merge comprise instructions that cause the data processor to apply the function to the value of the partially aggregated query result and the value of the related partially aggregated query result.
14 . The non-transitory computer-readable medium of claim 12 , wherein the plurality of instructions further comprise instructions that cause the data processor to:
detect a query for real-time data;
issue the query for real-time data on a persistent storage, wherein the persistent storage includes the plurality of partially aggregated query results;
determine a result of issuing the query on the persistent storage; and
determine an in-memory aggregation of the query for real-time data, wherein the complete aggregation of the query result is generated using the result of issuing the query on the persistent storage and the in-memory aggregation.
15 . The non-transitory computer-readable medium of claim 12 , wherein the plurality of instructions further comprise instructions that cause the data processor to:
receive a plurality of events in an event stream;
determine the plurality of events are out-of-order events;
determine a query result for each of the plurality of events; and
determine a partially aggregated query result based on the query result for each of the plurality of events.