Dictionary filtering and evaluation in columnar databases

1. A method comprising:

receiving a request to perform a query, the query comprising at least an operator and a request to return information about a value of interest in a columnar dataset stored on cloud storage, wherein

the query is associated with a filter generated to apply to the columnar dataset in response to the query,

at least one column in the columnar dataset is based on a dictionary, and

the dictionary maps one or more values for a column to one or more respective identifiers;

calculating a metric representing a likelihood that performing dictionary filtering will cause query regression based on at least a size of the dictionary and an estimated cost of evaluating the filter on the dictionary;

determining whether to perform dictionary filtering based on the metric by comparing the metric to a threshold; and

executing, responsive to the determination, one of:

performing dictionary filtering, the dictionary filtering comprising evaluating the filter on the dictionary before evaluating the filter on the column, and

retrieving the column associated with the dictionary and determining whether the value of interest is present in the column.

2. The method of claim 1 , wherein performing dictionary filtering further comprises eliminating a row group of the column.

3. The method of claim 2 , wherein the row group of the column is eliminated if all values in the row group fail the filter.

4. The method of claim 1 , wherein the metric increases based on the size of the dictionary increasing.

5. The method of claim 1 , wherein performing dictionary filtering further comprises:

monitoring query regression while dictionary filtering; and

aborting the dictionary filtering early in response to the query regression being above a threshold percentage.

6. The method of claim 1 , wherein calculating the metric based on at least the size of the dictionary and the estimated cost of evaluating the filter on the dictionary comprises calculating the metric with a machine learning model trained on data comprising at least a plurality of sizes of dictionaries and a plurality of estimated costs of evaluating filters on dictionaries.

7. The method of claim 1 , wherein the metric represents a likelihood that performing dictionary filtering will cause a decrease in central processing unit (CPU) efficiency for the query.

8. A system comprising:

a processor; and

a memory coupled with the processor and comprising stored instructions, the stored instructions executable by a processor and when executed cause the system to:

receive a request to perform a query, the query comprising at least an operator and a request to return information about a value of interest in a columnar dataset stored on cloud storage, wherein

the query is associated with a filter generated to apply to the columnar dataset in response to the query,

at least one column in the columnar dataset is based on a dictionary, and

the dictionary maps one or more values for a column to one or more respective identifiers;

calculate a metric representing a likelihood that performing dictionary filtering will cause query regression based on at least a size of the dictionary and an estimated cost of evaluating the filter on the dictionary;

determine whether to perform dictionary filtering based on the metric by comparison of the metric to a threshold; and

execute, responsive to the determination, one of:

performance of dictionary filtering by evaluating the filter on the dictionary before evaluating the filter on the column, and

retrieval of the column associated with the dictionary and determination of whether the value of interest is present in the column.

9. The system of claim 8 , wherein the stored instructions to execute performance of the dictionary filtering further comprise instructions that when executed cause the system to eliminate a row group of the column.

10. The system of claim 9 , wherein the row group of the column is eliminated if all values in the row group fail the filter.

11. The system of claim 8 , wherein the metric increases based on the size of the dictionary increasing.

12. The system of claim 8 , wherein the stored instructions to execute the performance of dictionary filtering further comprise instructions that when executed cause the system to:

monitor query regression while dictionary filtering; and

abort the dictionary filtering early when the query regression is above a threshold percentage.

13. The system of claim 8 , wherein the stored instruction to calculate the metric based on at least the size of the dictionary and the estimated cost of evaluating the filter on the dictionary comprise instructions that when executed cause the system to calculate the metric with a machine learning model trained on data comprising at least a plurality of sizes of dictionaries and a plurality of estimated costs of evaluating filters on dictionaries.

14. The system of claim 8 , wherein the metric represents a likelihood that performing dictionary filtering will cause a decrease in central processing unit (CPU) efficiency for the query.

15. A non-transitory computer readable medium comprising stored instructions encoded thereon that, when executed by a processor, cause at least one processor to:

receive a request to perform a query, the query comprising at least an operator and a request to return information about a value of interest in a columnar dataset stored on cloud storage, wherein

the query is associated with a filter generated to apply to the columnar dataset in response to the query,

at least one column in the columnar dataset is based on a dictionary, and

the dictionary maps one or more values for a column to one or more respective identifiers;

calculate a metric representing a likelihood that performing dictionary filtering will cause query regression based on at least a size of the dictionary and an estimated cost of evaluating the filter on the dictionary;

determining whether to perform dictionary filtering based on the metric by comparison of the metric to a threshold; and

execute, responsive to the determination, one of:

performance of dictionary filtering by evaluating the filter on the dictionary before evaluating the filter on the column, and

retrieval of the column associated with the dictionary and determination of whether the value of interest is present in the column.

16. The non-transitory computer readable medium of claim 15 , wherein the stored instructions to execute the performance of dictionary filtering further comprise stored instructions that when executed cause the processor to eliminate a row group of the column.

17. The non-transitory computer readable medium of claim 15 , wherein the metric increases based on the size of the dictionary increasing.

18. The non-transitory computer readable medium of claim 15 , wherein the stored instructions to execute the performance of the dictionary filtering further comprise stored instructions that when executed cause the processor to:

monitor query regression while dictionary filtering; and

abort the dictionary filtering early when the query regression is above a threshold percentage.

19. The non-transitory computer readable medium of claim 15 , wherein the stored instructions to calculate the metric based on at least the size of the dictionary and the estimated cost of evaluating the filter on the dictionary comprise instructions that when executed cause the processor to calculate the metric with a machine learning model trained on data comprising at least a plurality of sizes of dictionaries and a plurality of estimated costs of evaluating filters on dictionaries.

20. The non-transitory computer readable medium of claim 15 , wherein the metric represents a likelihood that performing dictionary filtering will cause a decrease in central processing unit (CPU) efficiency for the query.