Histogram with integrated distinct value sketches
Provided are systems and methods for creating histograms with distinct value sketches integrated therein and for query processing based on the histograms with distinct value sketches. In one example, the method may include storing a histogram that comprises a representation of a bucket of data from a database and that includes a distinct value sketch with a distinct value attribute that identifies an estimated number of distinct values within the bucket of data, receiving a database query, generating a query execution plan for the database query based on the distinct value attribute of the bucket within the distinct value sketch embedded within the histogram, and executing the database query on the bucket of data from the database based on the generated query execution plan.
1. A computing system comprising:
a database configured to store a data set; and
a processor configured to:
process a user input comprising a threshold number of distinct values allowed per bucket of a plurality of buckets;
estimate how many of the distinct values are included in the data set;
dynamically divide the data set into the plurality of buckets that have different size widths based on the estimated distinct values that are included within the data set and the threshold number of distinct values allowed per bucket,
receive a database query request associated with the data set;
estimate processing times for the database to process a database query corresponding to the database query request based on distinct values of data in the plurality of buckets that have different size widths; and
execute the database query on the data set within the database based on the estimated processing times.
2. The computing system of claim 1 , wherein the processor is configured to dynamically divide the data set into the plurality of buckets based on a density of the estimated distinct values within data set.
3. The computing system of claim 1 , wherein the processor is configured to dynamically divide the data set into the plurality of buckets based on skewed frequencies of distinct values within the data set.
4. The computing system of claim 1 , wherein the processor is configured to embed identifiers of the plurality of buckets and different respective bucket sizes of the plurality of buckets within a histogram, and execute the database query based on the histogram.
5. The computing system of claim 1 , wherein the processor is configured to iteratively add distinct values to a bucket from among the plurality of buckets until a predetermined threshold of distinct values are included in the bucket, and in response, add a next distinct value to a next bucket among the plurality of buckets.
6. The computing system of claim 1 , wherein the processor is configured to dynamically determine the different size widths of the plurality of buckets, respectively, such that each bucket has a number of distinct values that is equal to or less than the predefined threshold.
7. The computing system of claim 1 , wherein the processor is configured to generate a bucket that comprises an amount of distinct values that is equal to a predetermined threshold of distinct values, and generate a next bucket among the plurality of buckets with less than the predetermined threshold of distinct values therein.
8. A method comprising:
processing a user input comprising a threshold number of distinct values allowed per bucket of a plurality of buckets;
estimating how many of the distinct values are included in a data set stored in a database;
dynamically dividing the data set into the plurality of buckets that have different size widths based on the estimated distinct values that are included within the data set and the threshold number of distinct values allowed per bucket;
receiving a database query request associated with the data set;
estimating processing times for the database to process a database query corresponding to the database query request based on distinct values of data in the plurality of buckets that have different size widths; and
executing the database query on the data set within the database based on the estimated processing times.
9. The method of claim 8 , wherein the dynamically dividing comprises dividing the data set into the plurality of buckets based on a density of the estimated distinct values within data set.
10. The method of claim 8 , wherein the dynamically dividing comprises dynamically dividing the data set into the plurality of buckets based on skewed frequencies of distinct values within the data set.
11. The method of claim 8 , wherein the method comprises embedding identifiers of the plurality of buckets and different respective bucket sizes of the plurality of buckets within a histogram, and executing the database query based on the histogram.
12. The method of claim 8 , wherein the method comprises iteratively adding distinct values to a bucket from among the plurality of buckets until a predetermined threshold of distinct values are included in the bucket, and in response, adding a next distinct value to a next bucket among the plurality of buckets.
13. The method of claim 8 , wherein the dynamically determining comprises dynamically determining the different size widths of the plurality of buckets, respectively, such that each bucket has a number of distinct values that is equal to or less than the predefined threshold.
14. The method of claim 8 , wherein the dynamically dividing comprises generating a bucket that comprises an amount of distinct values that is equal to a predetermined threshold of distinct values, and generating a next bucket among the plurality of buckets with less than the predetermined threshold of distinct values therein.
15. A non-transitory computer-readable medium comprising instructions which when executed by a processor cause a computer to perform:
processing a user input comprising a threshold number of distinct values allowed per bucket of a plurality of buckets;
estimating how many of the distinct values are included in a data set stored in a database;
dynamically dividing the data set into the plurality of buckets that have different size widths based on the estimated distinct values that are included within the data set and the threshold number of distinct values allowed per bucket;
receiving a database query request associated with the data set;
estimating processing times for the database to process a database query corresponding to the database query request based on distinct values of data in the plurality of buckets that have different size widths; and
executing the database query on the data set within the database based on the estimated processing times.
16. The non-transitory computer-readable medium of claim 15 , wherein the dynamically dividing comprises dividing the data set into the plurality of buckets based on a density of the estimated distinct values within data set.
17. The non-transitory computer-readable medium of claim 15 , wherein the dynamically dividing comprises dynamically dividing the data set into the plurality of buckets based on skewed frequencies of distinct values within the data set.
18. The non-transitory computer-readable medium of claim 15 , wherein the computer is further configured to perform embedding identifiers of the plurality of buckets and different respective bucket sizes of the plurality of buckets within a histogram, and executing the database query based on the histogram.