Utilizing metadata to prune a data set
A query directed to database data stored across a set of files is received. The query includes predicates applied against metadata of the set of files. One or more files are removed from the set of files whose metadata does not satisfy a predicate of the plurality of predicates. Rows of the one or more remaining files of the set of files are filtered based on the one or more predicates.
1. A method comprising:
receiving a query directed to database data stored across a set of files, the query comprising one or more predicates;
removing, by a hardware resource, from the set of files, one or more files whose metadata does not satisfy a predicate of the plurality of predicates; and
filtering rows of one or more remaining files of the set of files based on the one or more predicates.
2. The method of claim 1 , further comprising:
executing the query against a remaining set of files to create a second set of files.
3. The method of claim 2 , further comprising:
returning the second set of files in response to the query.
4. The method of claim 1 , wherein removing from the set of files, each file whose metadata does not satisfy the predicate further comprises:
determining that the metadata for each file in the set of files does not correspond to ranges associated with a filter.
5. The method of claim 4 , wherein the filter comprises multiple disjoint range values.
6. The method of claim 1 , wherein each file of the set of files comprises a block of the database data.
7. The method of claim 1 , wherein the metadata for each file of the set of files comprises at least one of:
representative values of a block of the database data of the file;
a number of null values in the block of the database data of the file; or
maximum and minimum values of the block of the database data of the file.
8. The method of claim 1 , further comprising:
processing another data set of the query to generate a range bloom vector; and
applying the range bloom vector to the set of files to filter and prune the set of files.
9. The method of claim 1 , further comprising:
determining that a predicate of the plurality of predicates is not satisfied by the metadata of any of a remaining set of files; and
removing the predicate from the plurality of predicates.
10. A system comprising:
a hardware resource to:
receive a query directed to database data stored across a set of files, the query comprising one or more predicates;
remove, from the set of files, one or more files whose metadata does not satisfy a predicate of the plurality of predicates; and
filter rows of one or more remaining files of the set of files based on the one or more predicates.
11. The system of claim 10 , wherein the hardware resource is further to:
execute the query against a remaining set of files to create a second set of files.
12. The system of claim 11 , wherein the hardware resource is further to:
return the second set of files in response to the query.
13. The system of claim 10 , wherein to remove from the set of files, each file whose metadata does not satisfy the predicate, the hardware resource is further to:
determine that the metadata for each file in the set of files does not correspond to ranges associated with a filter.
14. The system of claim 13 , wherein the filter comprises a plurality of disjoint range values.
15. The system of claim 10 , wherein each file of the set of files comprises a block of the database data.
16. The system of claim 10 , wherein the metadata for each file of the set of files comprises at least one of:
representative values of a block of the database data of the file;
a number of null values in the block of the database data of the file; or
maximum and minimum values of the block of the database data of the file.
17. The system of claim 10 , wherein the hardware resource is further to:
process another data set of the query to generate a range bloom vector; and
apply the range bloom vector to the set of files to filter and prune the set of files.
18. The system of claim 10 , wherein the hardware resource is further to:
determine that a predicate of the plurality of predicates is not satisfied by the metadata of any of a remaining set of files; and
remove the predicate from the plurality of predicates.
19. A non-transitory computer-readable medium having instructions stored thereon that, when executed by a hardware resource, cause the hardware resource to:
receive a query directed to database data stored across a set of files, the query comprising one or more predicates;
remove, by the hardware resource, from the set of files, one or more files whose metadata does not satisfy a predicate of the plurality of predicates; and
filter rows of one or more remaining files of the set of files based on the one or more predicates.
20. The non-transitory computer-readable medium of claim 19 , wherein the hardware resource is further to:
process another data set of the query to generate a range bloom vector; and
apply the range bloom vector to the set of files to filter and prune the set of files.