Data compression for columnar databases into arbitrarily-sized persistent pages

1. A system, comprising:

at least one data processor; and

at least one memory storing instructions which, when executed by the at least one data processor, cause operations comprising:

generating, for a first data column included in a data chunk, a first dictionary enumerating, in a sorted order, a first set of unique values included in the first data column;

identifying, based at least on the first dictionary, a first compression technique for generating a first compressed representation of the first data column having a fewest quantity of bytes, the first compression technique including a dictionary compression applying the first dictionary and/or another compression technique;

generating a compressed data chunk by at least applying the first compression technique to compress the first data column included in the data chunk; and

storing, in a persistent page at a database, the compressed data chunk.

2. The system of claim 1 , wherein the persistent page is associated with a variable size, and wherein the persistent page is allocated based at least on a size of the first compressed representation of the first data column such that a size of the persistent page corresponds to that of the compressed data chunk.

3. The system of claim 1 , wherein the first compression technique comprises a null-value compression instead of the dictionary compression if the first dictionary includes a single NULL value.

4. The system of claim 1 , wherein the first compression technique comprises a single-value compression instead of the dictionary compression if the first dictionary includes a single non-NULL value.

5. The system of claim 1 , wherein the first compression technique comprises the dictionary compression if the first dictionary includes multiple non-NULL values and a first size of the first data column is greater than a second size of the first dictionary and a data vector generated by replacing each value in the first data column with a corresponding index value from the first dictionary.

6. The system of claim 1 , wherein the first compression technique comprises a prefix/suffix compression instead of or in addition to the dictionary compression if the first dictionary includes more than a threshold quantity of unique values with prefixes and/or suffixes.

7. The system of claim 1 , wherein the first compression technique comprises a run-length encoding instead of or in addition to the dictionary compression if the first data column includes less than a threshold quantity of runs of a same value and/or the first dictionary is less than a threshold size.

8. The system of claim 1 , wherein the first compression technique comprises a frame of reference compression instead of or in addition to the dictionary compression if the first dictionary includes less than a threshold quantity of outlier values relative to a range of values.

9. The system of claim 1 , wherein the database comprises a columnar database.

10. The system of claim 1 , wherein the operations further comprise:

generating, for a second data column included in the data chunk, a second dictionary enumerating, in the sorted order, a second set of unique values included in the second data column;

identifying, based at least on the second dictionary, a second compression technique for generating a second compressed representation of the second data column having the fewest quantity of bytes, the second compression technique including a dictionary compression applying the second dictionary and/or another compression technique; and

generating the compressed data chunk including by applying the second compression technique to compress the second data column included in the data chunk.

11. A computer-implemented method, comprising:

generating, for a first data column included in a data chunk, a first dictionary enumerating, in a sorted order, a first set of unique values included in the first data column;

identifying, based at least on the first dictionary, a first compression technique for generating a first compressed representation of the first data column having a fewest quantity of bytes, the first compression technique including a dictionary compression applying the first dictionary and/or another compression technique;

generating a compressed data chunk by at least applying the first compression technique to compress the first data column included in the data chunk; and

storing, in a persistent page at a database, the compressed data chunk.

12. The method of claim 11 , wherein the persistent page is associated with a variable size, and wherein the persistent page is allocated based at least on a size of the first compressed representation of the first data column such that a size of the persistent page corresponds to that of the compressed data chunk.

13. The method of claim 11 , wherein the first compression technique comprises a null-value compression instead of the dictionary compression if the first dictionary includes a single NULL value.

14. The method of claim 11 , wherein the first compression technique comprises a single-value compression instead of the dictionary compression if the first dictionary includes a single non-NULL value.

15. The method of claim 11 , wherein the first compression technique comprises the dictionary compression if the first dictionary includes multiple non-NULL values and a first size of the first data column is greater than a second size of the first dictionary and a data vector generated by replacing each value in the first data column with a corresponding index value from the first dictionary.

16. The method of claim 11 , wherein the first compression technique comprises a prefix/suffix compression instead of or in addition to the dictionary compression if the first dictionary includes more than a threshold quantity of unique values with prefixes and/or suffixes.

17. The method of claim 11 , wherein the first compression technique comprises a run-length encoding instead of or in addition to the dictionary compression if the first data column includes less than a threshold quantity of runs of a same value and/or the first dictionary is less than a threshold size.

18. The method of claim 11 , wherein the first compression technique comprises a frame of reference compression instead of or in addition to the dictionary compression if the first dictionary includes less than a threshold quantity of outlier values relative to a range of values.

19. The method of claim 11 , further comprising:

generating, for a second data column included in the data chunk, a second dictionary enumerating, in the sorted order, a second set of unique values included in the second data column;

identifying, based at least on the second dictionary, a second compression technique for generating a second compressed representation of the second data column having the fewest quantity of bytes, the second compression technique including a dictionary compression applying the second dictionary and/or another compression technique; and

generating the compressed data chunk including by applying the second compression technique to compress the second data column included in the data chunk.

20. A non-transitory computer readable medium storing instructions, which when executed by at least one data processor, result in operations comprising:

generating, for a first data column included in a data chunk, a first dictionary enumerating, in a sorted order, a first set of unique values included in the first data column;

identifying, based at least on the first dictionary, a first compression technique for generating a first compressed representation of the first data column having a fewest quantity of bytes, the first compression technique including a dictionary compression applying the first dictionary and/or another compression technique;

generating a compressed data chunk by at least applying the first compression technique to compress the first data column included in the data chunk; and

storing, in a persistent page at a database, the compressed data chunk.