Structured cluster execution for data streams

1. A method for executing a streaming query, comprising:

receiving, from a client device, a request to perform a streaming query on a data table, the streaming query specifying a set of operations on data of the data table, wherein the set of operations are performed using a set of operators;

determining an output mode for the streaming query as the append mode, the output mode specifying how results of the streaming query are written to an output table; and

responsive to determining that each operator of the set of operators can output results according to the append mode, performing, for each of one or more user-specified time increments:

receiving input data for the data table for a current time increment,

executing the set of operations for the streaming query based on at least the input data for the current time increment to generate incremental results for the set of operations,

writing the incremental results to an output table for the current time increment based on the append mode for the streaming query, wherein writing the incremental results comprises adding new rows to the output table without writing previous results of the set of operations for a previous time increment to the output table, and

providing the output table to the client device.

2. The method of claim 1 , further comprising:

receiving a second request to perform a second streaming query on the data table;

determining a second output mode for the second streaming query as an update mode; and

for each of one or more time increments, writing results to a second output table for the current time increment, comprising adding new rows to the second output table or outputting modification to one or more existing rows of the second output table that were output at previous time increments.

3. The method of claim 1 , further comprising:

receiving a second request to perform a second streaming query on the data table;

determining a second output mode for the second streaming query as a delta mode; and

for each of one or more time increments, writing results to a second output table for the current time increment, comprising adding new rows to the second output table or deleting one or more existing rows of the second output table that were output at previous time increments.

4. The method of claim 1 , wherein executing the set of operations for the streaming query further comprises:

generating an input table by combining previous input data for previous time increments and the input data for the current time increment; and

executing the set of operations for the streaming query on the input table.

5. The method of claim 1 , further comprising:

receiving a query state that is an output of executing the streaming query for previous time increments, the results for the current time increment generated based at least on the query state for the previous time increments.

6. The method of claim 1 , further comprising:

for the set of operations of the streaming query, generating a logical query plan including a set of ordered operators;

generating a plurality of physical query plans from the logical query plan, wherein a physical query plan includes one or more operators each associated with an operator input mode and an operator output mode; and

for each physical query plan, determining a cost function for executing the physical query plan, the cost function for the physical query plan based at least on the operator input modes and the operator output modes for the one or more operators of the physical query plan.

7. A non-transitory computer-readable storage medium comprising stored instructions executable by a processor, the instructions when executed causing the processor to:

receive, from a client device, a request to perform a streaming query on a data table, the streaming query specifying a set of operations on data of the data table, wherein the set of operations are performed using a set of operators;

determine an output mode for the streaming query as the append mode, the output mode specifying how results of the streaming query is written to an output table; and

responsive to determining that each operator of the set of operators can output results according to the append mode, performing, for each of one or more user-specified time increments, execute instructions that cause the processor to:

receive input data for the data table for a current time increment,

execute the set of operations for the streaming query based on at least the input data for the current time increment to generate incremental results for the set of operations,

write the incremental results to an output table for the current time increment based on the append mode for the streaming query, wherein writing the incremental results comprises instructions that cause the processor to add new rows to the output table without writing previous results of the set of operations for a previous time increment to the output table, and

provide the output table to the client device.

8. The non-transitory computer-readable storage medium of claim 7 , further comprising instructions that when executed cause the processor to,

receive a second request to perform a second streaming query on the data table;

determine a second output mode for the second streaming query as an update mode;

for each of one or more time increments, add new rows to a second output table or output modification to one or more existing rows of the second output table that were output at previous time increments.

9. The non-transitory computer-readable medium of claim 7 , further comprising instructions that when executed cause the processor to,

receive a second request to perform a second streaming query on the data table;

determine a second output mode for the second streaming query as a delta mode;

for each of one or more time increments, add new rows to a second output table or delete one or more existing rows of the second output table that were output at previous time increments.

10. The non-transitory computer-readable medium of claim 7 , further comprising instructions when executed causing the processor to:

generate an input table by combining previous input data for previous time increments and the input data for the current time increment; and

execute the set of operations for the streaming query on the input table.

11. The non-transitory computer-readable medium of claim 7 , the instructions when executed causing the processor to:

receive a query state that is an output of executing the streaming query for previous time increments, and

the results for the current time increment generated based at least on the query state for the previous time increments.

12. The non-transitory computer-readable medium of claim 7 , the instructions when executed causing the processor to:

generate, for the set of operations of the streaming query, a logical query plan including a set of ordered operators;

generate a plurality of physical query plans from the logical query plan, wherein a physical query plan includes one or more operators each associated with an operator input mode and an operator output mode; and

determine, for each physical query plan, a cost function for executing the physical query plan, the cost function for the physical query plan based at least on the operator input modes and the operator output modes for the one or more operators of the physical query plan.

13. A computer system, comprising:

a computer processor; and

a memory comprising stored instructions that when executed by the computer processor causes the computer system to:

receive, from a client device, a request to perform a streaming query on a data table, the streaming query specifying a set of operations on data of the data table, wherein the set of operations are performed using a set of operators;

determine an output mode for the streaming query as the append mode, the output mode specifying how results of the streaming query is written to an output table; and

responsive to determining that each operator of the set of operators can output results according to the append mode, performing, for each of one or more user-specified time increments:

receive input data for the data table for a current time increment,

execute the set of operations for the streaming query based on at least the input data for the current time increment to generate incremental results for the set of operations,

write the incremental results to an output table for the current time increment based on the append mode for the streaming query, wherein writing the incremental results comprises instructions that cause the processor to add new rows to the output table without writing previous results of the set of operations for a previous time increment to the output table, and

provide the output table to the client device.

14. The computer system of claim 13 , wherein the instructions further cause the processor to:

receive a second request to perform a second streaming query on the data table;

determine a second output mode for the second streaming query as an update mode;

for each of one or more time increments, the instructions when executed causing the computer system to add new rows to a second output table or output modification to one or more existing rows of the second output table that were output at previous time increments.

15. The computer system of claim 13 , wherein the instructions further cause the processor to:

receive a second request to perform a second streaming query on the data table;

determine a second output mode for the second streaming query as a delta mode;

for each of one or more time increments, the instructions when executed causing the computer system to add new rows to a second output table or delete one or more existing rows of the second output table that were output at previous time increments.

16. The computer system of claim 13 , the instructions when executed causing the computer system to:

generate an input table by combining previous input data for previous time increments and the input data for the current time increment; and

execute the set of operations for the streaming query on the input table.

17. The computer system of claim 13 , the instructions when executed causing the computer system to:

receive a query state that is an output of executing the streaming query for previous time increments, and

the results for the current time increment generated based at least on the query state for the previous time increments.