Query plans for analytic SQL constructs
Embodiments of the present disclosure relate to systems and methods for executing queries on a database platform. A processing device may execute a first operator in a query plan to process a set of data and generate an intermediate result of a query. the intermediate result of the first operator may be pushed, during execution of the query plan, to a plurality of secondary operators as the intermediate result is generated. Each of the plurality of secondary operators may be initiated to concurrently process the intermediate result to generate a plurality of second results, and a timing of processing of the intermediate result by one or more of the plurality of secondary operators is adjusted to coordinate the generation of the plurality of second results. The processor may execute the final operation on the plurality of second results to generate a final result.
1. A method comprising:
executing a first operator in a query plan to process a set of data and generate an intermediate result of a query;
pushing, by the first operator, during execution of the query plan, the intermediate result of the first operator to a plurality of secondary operators as the intermediate result is generated;
initiating, by the first operator, on one or more processors, each of the plurality of secondary operators to concurrently process the intermediate result to generate a plurality of second results, wherein beginning processing of the intermediate result by one or more of the plurality of secondary operators is delayed to coordinate the generation of the plurality of second results based on beginning a final consumption operation to be executed on the plurality of second results; and
executing the final operation on the plurality of second results to generate a final result.
2. The method of claim 1 , further comprising:
storing the final result to disk storage.
3. The method of claim 1 , wherein each of the plurality of secondary operators process the intermediate result with a different operation.
4. The method of claim 1 , wherein the intermediate result of the first operator is pushed to the plurality of secondary operators in response to determining that the first operator has finished processing the set of data.
5. The method of claim 1 , wherein the intermediate result generated by the first operator is not recomputed by any of the plurality of secondary operators, and wherein the intermediate result is processed by the plurality of secondary operators to execute a plurality of different queries.
6. The method of claim 1 , wherein the intermediate result is not materialized.
7. The method of claim 1 , wherein the intermediate result is not recomputed.
8. The method of claim 1 , wherein the intermediate result is not stored in disk storage.
9. The method of claim 1 , wherein the final operation is a join operation.
10. A system comprising:
a memory; and
one or more processors, operatively coupled to the memory, the one or more processors to:
execute a first operator in a query plan to process a set of data and generate an intermediate result of a query;
push, during execution of the query plan, the intermediate result of the first operator to a plurality of secondary operators as the intermediate result is generated;
initiate, by the first operator, each of the plurality of secondary operators to concurrently process the intermediate result to generate a plurality of second results, wherein beginning processing of the intermediate result by one or more of the plurality of secondary operators is delayed to coordinate the generation of the plurality of second results based on beginning a final consumption operation to be executed on the plurality of second results; and
execute the final operation on the plurality of second results to generate a final result.
11. The system of claim 10 , wherein the one or more processors are further to:
store the final result to disk storage.
12. The system of claim 10 , wherein each of the plurality of secondary operators process the intermediate result with a different operation.
13. The system of claim 10 , wherein the one or more processors push the intermediate result of the first operator to the plurality of secondary operators in response to determining that the first operator has finished processing the set of data.
14. The system of claim 10 , wherein the intermediate result generated by the first operator is not recomputed by any of the plurality of secondary operators, and wherein the intermediate result is processed by the plurality of secondary operators to execute a plurality of different queries.
15. The system of claim 10 , wherein the intermediate result is not materialized.
16. The system of claim 10 , wherein the intermediate result is not recomputed.
17. The system of claim 10 , wherein the intermediate result is not stored in disk storage.
18. The system of claim 10 , wherein the final operation is a join operation.
19. A non-transitory computer-readable medium having instructions stored thereon which, when executed by one or more processors, cause the one or more processors to:
execute a first operator in a query plan to process a set of data and generate an intermediate result of a query;
push, during execution of the query plan, the intermediate result of the first operator to a plurality of secondary operators as the intermediate result is generated;
initiate, by the first operator, each of the plurality of secondary operators to concurrently process the intermediate result to generate a plurality of second results, wherein beginning processing of the intermediate result by one or more of the plurality of secondary operators is delayed to coordinate the generation of the plurality of second results based on beginning a final consumption operation to be executed on the plurality of second results; and
execute the final operation on the plurality of second results to generate a final result.
20. The non-transitory computer-readable medium of claim 19 , wherein the one or more processors are further to:
store the final result to disk storage.
21. The non-transitory computer-readable medium of claim 19 , wherein each of the plurality of secondary operators process the intermediate result with a different operation.
22. The non-transitory computer-readable medium of claim 19 , wherein the one or more processors push the intermediate result of the first operator to the plurality of secondary operators in response to determining that the first operator has finished processing the set of data.
23. The non-transitory computer-readable medium of claim 19 , wherein the intermediate result generated by the first operator is not recomputed by any of the plurality of secondary operators, and wherein the intermediate result is processed by the plurality of secondary operators to execute a plurality of different queries.
24. The non-transitory computer-readable medium of claim 19 , wherein the intermediate result is not materialized.
25. The non-transitory computer-readable medium of claim 19 , wherein the intermediate result is not recomputed.
26. The non-transitory computer-readable medium of claim 19 , wherein the intermediate result is not stored in disk storage.
27. The non-transitory computer-readable medium of claim 19 , wherein the final operation is a join operation.