Techniques of heterogeneous hardware execution for SQL analytic queries for high volume data processing
The present invention relates to optimized access of a database. Herein are techniques to accelerate execution of any combination of ad hoc query, heterogenous hardware, and fluctuating workload. In an embodiment, a computer receives a data access request for data tuples and compiles the data access request into relational operators. A particular implementation of a particular relational operator is dynamically selected from multiple interchangeable implementations. Each interchangeable implementation contains respective physical operators. A particular hardware operator for a particular physical operator is selected from multiple interchangeable hardware operators that include: a first hardware operator that executes on first processing hardware, and a second hardware operator that executes on second processing hardware that is functionally different from the first processing hardware. A response to the data access request is generated based on: the data tuples, the particular implementation of the particular relational operator, and the particular hardware operator.
1. A method comprising:
generating a query plan for a database statement that references a plurality of columns in a set of relational tuples, wherein the query plan includes a hardware-independent rotate operator;
executing the query plan, wherein executing the query plan includes rotating, by a graphical processing unit (GPU), the plurality of columns; and
generating and sending a response for the database statement that is based on said rotating the plurality of columns;
wherein the method is performed by one or more computers.
2. The method of claim 1 wherein the hardware-independent rotate operator is undefined in at least one selected from a group consisting of: a relational algebra and a query language.
3. The method of claim 1 wherein a database cursor is not used by at least one selected from a group consisting of: the database statement and the query plan.
4. The method of claim 1 wherein the database statement does not contain a sub query.
5. The method of claim 1 wherein said generating the query plan does not comprise dynamically generating a query language query.
6. The method of claim 1 wherein said rotating does not use a pivot column.
7. The method of claim 1 wherein the hardware-independent rotate operator is configurable for at least one selected from the group consisting of:
a rotation direction, an amount of rotation, and a count of quarter turns.
8. The method of claim 1 further comprising:
detecting that an output format of an upstream operator needs conversion to a format selected from the group consisting of: an input format of a downstream operator and an output file format; and
selecting, in response to said detecting, the hardware-independent rotate operator.
9. The method of claim 1 further comprising transferring output of said rotating to a downstream operator that performs vertical slicing or horizontal slicing.
10. A method comprising:
generating a query plan for a database statement that references a plurality of columns in a set of relational tuples, wherein:
the database statement does not specify transposition,
a matrix comprises the set of relational tuples, and
the query plan includes a hardware-independent transpose operator;
executing the query plan, wherein executing the query plan includes transposing, by a graphical processing unit (GPU), the plurality of columns without changing values in a diagonal of the matrix; and
generating and sending a response for the database statement that is based on said transposing;
wherein the method is performed by one or more computers.
11. The method of claim 8 wherein
said transposing includes accessing said plurality of columns in a form selected from the group consisting of: column major form and row major form.
12. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause:
generating a query plan for a database statement that references a plurality of columns in a set of relational tuples, wherein
the query plan includes a hardware-independent rotate operator;
executing the query plan, wherein executing the query plan includes rotating, by a graphical processing unit (GPU), the plurality of columns; and
generating and sending a response for the database statement that is based on said rotating the plurality of columns.
13. The one or more non-transitory computer-readable storage media of claim 12 wherein the hardware-independent rotate operator is undefined in at least one selected from a group consisting of: a relational algebra and a query language.
14. The one or more non-transitory computer-readable storage media of claim 12 wherein a database cursor is not used by at least one selected from a group consisting of: the database statement and the query plan.
15. The one or more non-transitory computer-readable storage media of claim 12 wherein the database statement does not contain a subquery.
16. The one or more non-transitory computer-readable storage media of claim 12 wherein said rotating does not use a pivot column.
17. The one or more non-transitory computer-readable storage media of claim 12 wherein the hardware-independent rotate operator is configurable for at least one selected from the group consisting of: a rotation direction, an amount of rotation, and a count of quarter turns.
18. The one or more non-transitory computer-readable storage media of claim 12 wherein the instructions further cause:
detecting that an output format of an upstream operator needs conversion to a format selected from the group consisting of: an input format of a downstream operator and an output file format; and
selecting, in response to said detecting, the hardware-independent rotate operator.
19. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause:
generating a query plan for a database statement that references a plurality of columns in a set of relational tuples, wherein:
the database statement does not specify transposition,
a matrix comprises the set of relational tuples, and
the query plan includes a hardware-independent transpose operator;
executing the query plan, wherein executing the query plan includes transposing, by a graphical processing unit (GPU), the plurality of columns without changing values in a diagonal of the matrix; and
generating and sending a response for the database statement that is based on said transposing.
20. The one or more non-transitory computer-readable storage media of claim 19 wherein
said transposing includes accessing said plurality of columns in a form selected from the group consisting of: column major form and row major form.