QUERY PLANS FOR ANALYTIC SQL CONSTRUCTS
A system and method for managing data storage and data access with querying data in a distributed system without buffering the results of intermediate operations in storage. The method includes providing, by a first processor, as a result of executing a query plan, an intermediate result to a plurality of secondary processors. Processing of the intermediate result by the plurality of secondary processors generates a plurality of secondary results. A final result is stored to a storage device.
1 . A system comprising:
a memory; and
a first processor and a second set of processors of a computing device, operatively coupled to the memory, the computing device to:
initiate, within the first processor, a first parallel execution process in a query plan to process a set of data and generate an intermediate result of a query;
provide, by the first processor, during execution of the query plan, the intermediate result to the second set of processors for processing by a plurality of secondary parallel execution processes within the second set of processors;
initiate, by the second set of processors, the plurality of secondary parallel execution processes to process the intermediate result to generate a plurality of secondary results; and
store a final result to a storage device.
2 . The system of claim 1 wherein the intermediate result and the plurality of second results is not materialized.
3 . The system of claim 1 , wherein the system generates a final consumption operation without writing the plurality of second results to the storage device.
4 . The system of claim 1 , wherein the query plan comprises at least one of:
a grouping set;
a roll-up; or
a cube aggregate.
5 . The system of claim 1 , wherein each of the plurality of secondary parallel execution processes are different from one another.
6 . The system of claim 1 , wherein a final consumption operation is delayed until a completion of the plurality of secondary parallel execution processes.
7 . The system of claim 1 , wherein the one or more processors generate a final consumption operation without writing the plurality of second results to the storage device.
8 . The system of claim 1 , wherein the query plan comprises a window aggregate.
9 . The system of claim 1 , wherein the first parallel execution process pushes the intermediate result to the plurality of secondary parallel execution processes without buffering the intermediate result.
10 . The system of claim 1 , wherein a final consumption operation delays until a completion of the plurality of secondary parallel execution processes.
11 . A method comprising:
initiating, within a first processor, a first parallel execution process in a query plan to process a set of data and generate an intermediate result of a query;
providing, by the first processor, during execution of the query plan, the intermediate result to a second set of processors for processing by a plurality of secondary parallel execution processes within the second set of processors;
initiating, by the second set of processors, the plurality of secondary parallel execution processes to process the intermediate result to generate a plurality of secondary results; and
storing a final result to a storage device.
12 . The method of claim 11 , wherein the plurality of second results is generated without writing the intermediate result to the storage device.
13 . The method of claim 11 , wherein a final consumption operation is generated without writing the plurality of second results to the storage device.
14 . The method of claim 11 , wherein the intermediate result is not materialized.
15 . The method of claim 11 , wherein the plurality of second results is not materialized.
16 . The method of claim 11 , wherein the query plan comprises at least one of:
a grouping set;
a roll-up; or
a cube aggregate.
17 . The method of claim 11 , wherein each of the plurality of secondary parallel execution processes are different from one another.
18 . The method of claim 11 , wherein a final consumption operation is delayed until a completion of the plurality of secondary parallel execution processes.
19 . A non-transitory machine-readable medium storing instructions which, when executed by a first processor and a second set of processors of a computing device, cause the computing device to:
initiate, within the first processor, a first parallel execution process in a query plan to process a set of data and generate an intermediate result of a query;
provide, by the first processor, during execution of the query plan, the intermediate result to the second set of processors for processing by a plurality of secondary parallel execution processes within the second set of processors;
initiate, by the second set of processors, the plurality of secondary parallel execution processes to process the intermediate result to generate a plurality of secondary results; and
store a final result to a storage device.
20 . The non-transitory machine-readable medium of claim 19 , wherein the instructions further cause the second set of processors to generate the plurality of second results without writing the intermediate result to the storage device.
21 . The non-transitory machine-readable medium of claim 19 , wherein the instructions further cause the first processor to generate a final consumption operation without writing the plurality of second results to the storage device.
22 . The non-transitory machine-readable medium of claim 19 , wherein the intermediate result is not materialized.
23 . The non-transitory machine-readable medium of claim 19 , wherein the query plan comprises a window aggregate.
24 . The non-transitory machine-readable medium of claim 19 , wherein the instructions further cause the first parallel execution process to push the intermediate result to the plurality of secondary parallel execution processes without buffering the intermediate result.
25 . The non-transitory machine-readable medium of claim 19 , wherein the instructions further cause a delay of a final consumption operation until a completion of the plurality of secondary parallel execution processes.