Configurable data windows for archived relations
Techniques for managing configurable data windows for streams and/or archived relations are provided. In some examples, a query that identifies at least a data stream or an archived relation may be identified. Additionally, a configurable window size may be identified. In some instances, the query may be initialized with at least a portion of the historical data based at least in part on the window size. Further, the query may be evaluated based at least in part on the data stream and the portion of the historical data.
1. A system, comprising:
a memory storing a plurality of instructions; and
one or more processors that access the memory, wherein the one or more processors execute the plurality of instructions to at least:
identify a continuous query that identifies a data stream;
identify a configurable window size;
generate an archived relation to include at least a portion of historical data and incoming events of the identified data stream that are within a window that corresponds to the configurable window size, the historical data from a previous time associated with the identified data stream;
initialize the continuous query with the archived relation to process at least the portion of the historical data based at least in part on the window size; and
evaluate the archived relation using the continuous query based at least in part on the data stream and the portion of the historical data.
2. The system of claim 1 , wherein the continuous query processes incoming real-time business event data of the data stream.
3. The system of claim 1 , wherein the evaluation of the archived relation includes at least applying the continuous query against the data stream based at least in part on the configurable window size.
4. The system of claim 1 , wherein the data stream comprises a relation configured with a second configurable window size for referencing a portion of the data stream within a second window with a size based at least in part on a second configurable window size.
5. The system of claim 4 , wherein the second window moves along the data stream with time.
6. The system of claim 1 , wherein the historical data is not included in the data stream at a time after initializing the query.
7. The system of claim 1 , wherein the historical data comprises business event data from the data stream at a time before initializing the query.
8. The system of claim 1 , wherein evaluating the archived relation includes at least forming a query graph with nodes representing operators of the continuous query.
9. The system of claim 8 , wherein at least one of the nodes identifies the window size.
10. The system of claim 8 , wherein the one or more processors are further configured to execute the plurality of instructions to at least traverse the query graph topographically from a source.
11. The system of claim 10 , wherein the one or more processors further execute the plurality of instructions to at least initialize the continuous query with at least the historical data at a lowest stateful operator identified in the query graph.
12. A computer-readable memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising:
instructions that cause the one or more processors to receive a continuous query configured to process archived relations, the continuous query including at least one of an identifier of the data stream or an identifier of historical data;
instructions that cause the one or more processors to generate a query graph based at least in part on the continuous query;
instructions that cause the one or more processors to receive a window size from an entity associated with the data stream;
instructions that cause the one or more processors to generate an archived relation to include at least a portion of the historical data and incoming events of the data steam that are within a window that corresponds to the window size, the historical data from a previous time associated with the data stream;
instructions that cause the one or more processors to initialize the continuous query with the archived relation to process at least the portion of the historical data based at least in part on the window size; and
instructions that cause the one or more processors to evaluate the archived relation using the continuous query based at least in part on the historical data, the window size being based at least in part on the query graph.
13. The computer-readable memory of claim 12 , wherein the archived relation is identified by a data definition language annotation of the continuous query.
14. The computer-readable memory of claim 13 , wherein the data definition language annotation identifies at least one of a location of the historical data, a source of the archived relation, a data object associated with the archived relation, operation information associated with processing the continuous query, one or more relevant columns of a database corresponding to the historical data, a data object corresponding to output of the continuous query, or a location for providing the output of the continuous query.
15. The computer-readable memory of claim 12 , wherein the plurality of instructions further comprise instructions that cause the one or more processors to determine an amount of historical data with which to initialize the continuous query based at least in part on the window size.
16. The computer-readable memory of claim 12 , wherein the continuous query is initialized with the at least a portion of the historical data at a stateful operator of the query graph while traversing the query graph topographically from a source of the query graph.
17. A computer-implemented method, comprising:
receiving a continuous query configured to process archived relations associated with business event data, the continuous query including an identifier of historical data associated with the stream;
generating an archived relation to include at least a portion of the historical data and incoming events of the steam that are within a window that corresponds to the window size, the historical data from a previous time associated with the stream;
receiving a window size configured to identify a bounded range of the archived relation to-process;
generating a query graph based at least in part on the continuous query, the query graph including at least a node representing an operator of the continuous query;
traversing the query graph topographically from a source node of the query graph to identify a stateful operator;
initializing the continuous query with the archived relation to process at least the portion of the historical data at the identified stateful operator based at least in part on the window size; and
evaluating the archived relation using the continuous query based at least in part on the window size.
18. The computer-implemented method of 17 , wherein the historical data includes business event data from a point in time of the archived relation prior to initializing the continuous query.
19. The computer-implemented method of 17 , further comprising determining an optimal amount of the historical data with which to initialize the continuous query based at least in part on the operator of the continuous query.
20. The computer-implemented method of claim 17 , further comprising determining an optimal amount of the historical data with which to initialize the continuous query based at least in part on the window size.