Many-core algorithms for in-memory column store databases
A pattern can be identified in at least part of a query whose definition is received in a query request. The identified pattern can be matched with a set of pre-defined patterns, each of which has associated therewith at least one pre-compiled query execution sub-component of a plurality of pre-compiled query execution sub-components retained in a library. A plan for executing the query can be generated, for example by incorporating the pre-compiled query execution sub-component associated with the matched pattern into the plan based on a pseudo code representation of the plan derived from the definition.
1. A non-transitory computer program product storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:
receiving a query request comprising a definition of a query of a database persisted in a column-based storage;
identifying a pattern in at least part of the query;
matching the identified pattern with a set of pre-defined patterns, each of the pre-defined patterns having associated therewith at least one pre-compiled query execution sub-component of a plurality of pre-compiled query execution sub-components retained in a library;
selecting, based at least on the matching of identified patterns, an optimal sequence for processing a plurality of tables that must be joined to respond to the query, the optimal sequence avoids intermediate results to at least enable the query to be executed in a single pass of the database;
generating a plan for executing the query, the generating of the plan comprising incorporating, into the plan, the optimal sequence for processing the plurality of tables, and the generating of the plan further comprising incorporating, into the plan, the pre-compiled query execution sub-component associated with the matched pattern into the plan based on a pseudo code representation of the plan derived from the definition; and
executing the query using the generated plan.
2. A computer program product as in claim 1 , wherein the operations further comprise deriving the pseudo code representation of the plan from the definition.
3. A computer program product as in claim 1 , wherein the generating further comprises creating a single function to call the pre-compiled query execution component and the one or more other pre-compiled query execution components to generate the plan.
4. A computer program product as in claim 3 , wherein the single function defines a desired result and accesses a predefined parallelization plan from a set of two or more predefined parallelization plans based at least in part of the matching of the identified pattern.
5. A computer program product as in claim 1 , wherein the pre-compiled query execution sub-component comprises one or more pre-compiled SQL operations expressed in C++.
6. A system comprising:
at least one programmable processor; and
a machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one programmable processor to perform operations comprising:
receiving a query request comprising a definition of a query of a database persisted in a column-based storage;
identifying a pattern in at least part of the query;
matching the identified pattern with a set of pre-defined patterns, each of the pre-defined patterns having associated therewith at least one pre-compiled query execution sub-component of a plurality of pre-compiled query execution sub-components retained in a library;
selecting, based at least on the matching of identified patterns, an optimal sequence for processing a plurality of tables that must be joined to respond to the query, the optimal sequence avoids intermediate results to at least enable the query to be executed in a single pass of the database;
generating a plan for executing the query, the generating of the plan comprising incorporating, into the plan, the optimal sequence for processing the plurality of tables, and the generating of the plan further comprising incorporating, into the plan, the pre-compiled query execution sub-component associated with the matched pattern into the plan based on a pseudo code representation of the plan derived from the definition; and
executing the query using the generated plan.
7. A system as in claim 6 , wherein the operations further comprise deriving the pseudo code representation of the plan from the definition.
8. A system as in claim 6 , wherein the generating further comprises creating a single function to call the pre-compiled query execution component and the one or more other pre-compiled query execution components to generate the plan.
9. A system as in claim 8 , wherein the single function defines a desired result and accesses a predefined parallelization plan from a set of two or more predefined parallelization plans based at least in part of the matching of the identified pattern.
10. A system as in claim 6 , wherein the pre-compiled query execution sub-component comprises one or more pre-compiled SQL operations expressed in C++.
11. A computer-implemented method comprising:
receiving a query request comprising a definition of a query of a database persisted in a column-based storage;
identifying a pattern in at least part of the query;
matching the identified pattern with a set of pre-defined patterns, each of the pre-defined patterns having associated therewith at least one pre-compiled query execution sub-component of a plurality of pre-compiled query execution sub-components retained in a library;
selecting, based at least on the matching of identified patterns, an optimal sequence for processing a plurality of tables that must be joined to respond to the query, the optimal sequence avoids intermediate results to at least enable the query to be executed in a single pass of the database;
generating a plan for executing the query, the generating of the plan comprising incorporating, into the plan, the optimal sequence for processing the plurality of tables, and the generating of the plan further comprising incorporating, into the plan, the pre-compiled query execution sub-component associated with the matched pattern into the plan based on a pseudo code representation of the plan derived from the definition; and
executing the query using the generated plan.
12. A computer-implemented method as in claim 11 , further comprising deriving the pseudo code representation of the plan from the definition.
13. A computer-implemented method as in claim 11 , wherein the generating further comprises creating a single function to call the pre-compiled query execution component and the one or more other pre-compiled query execution components to generate the plan.
14. A computer-implemented method as in claim 13 , wherein the single function defines a desired result and accesses a predefined parallelization plan from a set of two or more predefined parallelization plans based at least in part of the matching of the identified pattern.
15. A computer-implemented method as in claim 11 , wherein the pre-compiled query execution sub-component comprises one or more pre-compiled SQL operations expressed in C++.
16. A computer-implemented method as in claim 11 , wherein at least one of the receiving, the identifying, the matching, the generating, and the executing is performed by at least one programmable processor.