Progressive query computation using streaming architectures
The described implementations relate to processing of electronic data. One implementation is manifest as a technique that can include obtaining a relational query that references one or more data items and associating progress intervals with the data items. The technique can also include converting the relational query into a corresponding streaming query, and providing the streaming query and the data items with the progress intervals to a stream engine that produces incremental results of the query. For example, the progress intervals can be based on row numbers of a relational database table. The progress intervals can be used to define event lifetimes of streaming events that are provided as inputs to the stream engine.
1. A method comprising:
obtaining a relational query that references relational data items from a relational data table that lacks an explicit temporal attribute for ordering the relational data items of the relational data table; and
adapting the relational data items for processing by a stream engine by associating explicit temporal data with the relational data items, the explicit temporal data comprising different lifetimes.
2. The method of claim 1 , further comprising:
receiving code; and
processing the code to derive the relational query.
3. The method of claim 1 , further comprising:
assigning the different lifetimes to stream events processed by the stream engine, the stream events corresponding to individual rows of the relational data table.
4. The method of claim 3 , further comprising:
creating the different lifetimes based at least on row numbers of the relational data table.
5. The method of claim 4 , wherein the different lifetimes include different end times for at least some of the relational data items.
6. The method of claim 1 , further comprising:
grouping multiple rows of data items together into a first set of multiple first rows of data items and a second set of multiple second rows of data items; and
adapting the multiple rows of data items for processing by the stream engine by:
assigning a first progress interval to the first set of multiple first rows; and
assigning a second progress interval to the second set of multiple second rows.
7. A system comprising:
one or more processors; and
one or more computer-readable storage media comprising instructions which, when executed by the one or more processors, cause the one or more processors to:
access relational data items from a relational data table that lacks an explicit temporal attribute for ordering the relational data items of the relational data table; and
adapt the relational data items for processing by a stream engine by associating explicit temporal data with the relational data items.
8. The system of claim 7 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
adapt the relational data items by defining different lifetimes for different sets of relational data items.
9. The system of claim 8 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
assign different begin lifetimes to each set of relational data items; and
assign a common end lifetime to the different sets of relational data items.
10. The system of claim 7 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
identify a partitioning attribute, other than a row number, of a query that references the relational data items;
group the relational data items into different sets of relational data items, each relational data item in a given set sharing the same value for the partitioning attribute; and
use the partitioning attribute as a progress attribute for the stream engine.
11. The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
use the partitioning attribute as the progress attribute by assigning different lifetimes to each set of relational data items.
12. The system of claim 11 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
assign different begin lifetimes to each set of relational data items.
13. The system of claim 12 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
assign different end lifetimes to each set of relational data items.
14. The system of claim 13 , wherein the instructions, when executed by the one or more processors, cause the one or more processors to:
assign a common end lifetime for results provided by the stream engine when processing the relational data items.
15. A hardware computer-readable storage media storing instructions which, when executed by one or more hardware processors, cause the one or more hardware processors to perform acts comprising:
obtaining a relational query that references relational data items from a relational data table that lacks an explicit temporal attribute for ordering the relational data items of the relational data table; and
adapting the relational data items for processing by a stream engine by associating explicit temporal data with the relational data items.
16. The hardware computer-readable storage media of claim 15 , the explicit temporal data defining lifetime windows for the stream engine to process the relational data items.
17. The hardware computer-readable storage media of claim 16 , the lifetime windows having different begin lifetimes and a shared end lifetime.
18. The hardware computer-readable storage media of claim 17 , the shared end lifetime being infinity.
19. The hardware computer-readable storage media of claim 17 , the acts further comprising:
setting a first begin lifetime for multiple first rows of the relational data items; and
setting a second begin lifetime for multiple second rows of the relational data items.
20. The hardware computer-readable storage media of claim 15 , the acts further comprising:
deriving the relational query from script code.