Cache conscious techniques for generation of quasi-dense grouping codes of compressed columnar data in relational database systems
Herein are techniques for dynamic aggregation of results of a database request, including concurrent grouping of result items in memory based on quasi-dense keys. Each of many computational threads concurrently performs as follows. A hash code is calculated that represents a particular natural grouping key (NGK) for an aggregate result of a database request. Based on the hash code, the thread detects that a set of distinct NGKs that are already stored in the aggregate result does not contain the particular NGK. A distinct dense grouping key for the particular NGK is statefully generated. The dense grouping key is bound to the particular NGK. Based on said binding, the particular NGK is added to the set of distinct NGKs in the aggregate result.
1. A method comprising:
concurrently by each thread of a plurality of computational threads:
calculating a hash code that represents a particular natural grouping key (NGK) for an aggregation needed for a database request;
detecting, based on the hash code, that a grouping key mapping for the aggregation does not contain the particular NGK;
statefully generating a distinct dense grouping key for the particular NGK;
binding, in the grouping key mapping, the dense grouping key with the particular NGK; and
generating a result for the database request based on the grouping key mapping.
2. The method of claim 1 wherein the dense grouping key comprises at least one selected from a group consisting of: a number, and a next value in a monotonically increasing sequence.
3. The method of claim 2 wherein the monotonically increasing sequence is discontinuous.
4. The method of claim 1 further comprising immediately after executing the database request, discarding a grouping key mapping.
5. The method of claim 1 wherein at least one selected from a group consisting of:
a) said grouping key mapping provides a bidirectional mapping between natural grouping keys and dense grouping keys, and
b) a grouping key mapping comprises at least one selected from a group consisting of: an indexer tree and two trees.
6. The method of claim 5 wherein:
the indexer tree contains a one or more tree nodes that include a particular node as a root of the indexer tree;
each tree leaf of the one or more tree nodes contains a segment of a segmented array.
7. The method of claim 6 wherein the indexer tree retains the particular node when the particular node does not remain the root of the indexer tree.
8. The method of claim 6 wherein the segment of said segmented array contains at least one selected from a group consisting of:
an array of pointers, and
a same amount of bytes.
9. The method of claim 8 wherein each pointer of the array of pointers contains more bytes than the dense grouping key.
10. The method of claim 6 wherein each segment of the segmented array has a same memory alignment as a hardware cache line.
11. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause:
concurrently by each thread of a plurality of computational threads:
calculating a hash code that represents a particular natural grouping key (NGK) for an aggregation needed for a database request;
detecting, based on the hash code, that a grouping key mapping for the aggregation does not contain the particular NGK;
statefully generating a distinct dense grouping key for the particular NGK;
binding, in a grouping key mapping, the dense grouping key with the particular NGK; and
generating a result for the database request based on a grouping key mapping.
12. The one or more non-transitory computer-readable media of claim 11 wherein the dense grouping key comprises at least one selected from a group consisting of: a number, and a next value in a monotonically increasing sequence.
13. The one or more non-transitory computer-readable media of claim 12 wherein the monotonically increasing sequence is discontinuous.
14. The one or more non-transitory computer-readable media of claim 11 wherein the instructions further cause immediately after executing the database request, discarding a grouping key mapping.
15. The one or more non-transitory computer-readable media of claim 11 wherein at least one selected from a group consisting of:
a) said grouping key mapping provides a bidirectional mapping between natural grouping keys and dense grouping keys, and
b) a grouping key mapping comprises at least one selected from a group consisting of: an indexer tree and two trees.
16. The one or more non-transitory computer-readable media of claim 15 wherein:
the indexer tree contains a one or more tree nodes that include a particular node as a root of the indexer tree;
each tree leaf of the one or more tree nodes contains a segment of a segmented array.
17. The one or more non-transitory computer-readable media of claim 16 wherein the indexer tree retains the particular node when the particular node does not remain the root of the indexer tree.
18. The one or more non-transitory computer-readable media of claim 16 wherein the segment of said segmented array contains at least one selected from a group consisting of:
an array of pointers, and
a same amount of bytes.
19. The one or more non-transitory computer-readable media of claim 18 wherein each pointer of the array of pointers contains more bytes than the dense grouping key.
20. The one or more non-transitory computer-readable media of claim 16 wherein each segment of the segmented array has a same memory alignment as a hardware cache line.