Bloom filter for avoiding partition loading
Embodiments are described for a database management system comprising a memory and at least one processor coupled to the memory. The at least one processor is configured to receive a query that corresponds to a data slice and determine a bloom filter based on the query. The at least one processor is further configured to determine that the data slice includes data requested by the query based on the bloom filter and in response to determining that the data slice includes the data requested by the query, load the data slice to the memory.
1 . A database management system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
receive a query to retrieve data that corresponds to a data slice, wherein the query includes one or more identifiers, wherein the one or more identifiers correspond to one or more data properties included in the data slice;
determine whether a bloom filter includes the one or more identifiers based on the query;
in response to determining that the bloom filter includes the one or more identifiers, determine that the data slice includes the data requested by the query based on the bloom filter; and
in response to determining that the data slice includes the data requested by the query, load the data slice to the memory.
2 . The database management system of claim 1 , wherein the at least one processor is further configured to:
receive a second query that corresponds to a second data slice;
determine that no bloom filter is available for the second query; and
in response to determining that no bloom filter is available, load the second data slice to the memory.
3 . The database management system of claim 2 , wherein to determine that no bloom filter is available, the at least one processor is further configured to determine that no bloom filter corresponds to the second data slice.
4 . The database management system of claim 2 , wherein to determine that no bloom filter is available, the at least one processor is further configured to:
determine that a second bloom filter corresponds to the data slice;
determine that the second query indicates an identifier; and
determine that the second bloom filter does not correspond to the identifier.
5 . The database management system of claim 4 , wherein the at least one processor is further configured to:
in response to determining that the second bloom filter does not correspond to the identifier, add 1 to a bloom filter counter of the second bloom filter;
determine that the bloom filter counter is more than a threshold; and
invalidate the bloom filter.
6 . The database management system of claim 5 , wherein the at least one processor is further configured to create a new bloom filter for the second data slice, wherein the new bloom filter corresponds to at least the identifier.
7 . The database management system of claim 1 , wherein the at least one processor is further configured to:
receive a further query corresponding to a further data slice;
determine a further bloom filter based on the further query;
determine that the further data slice does not include data requested by the further query based on the further bloom filter; and
in response to determining that the further data slice does not include the data requested by the further query, refrain from loading the further data slice to the memory.
8 . The database management system of claim 1 , wherein the at least one processor is further configured to:
determine a change on the data slice within a predetermined time period; and
invalidate the bloom filter.
9 . A computer-implemented method for loading operations, comprising:
receiving a query to retrieve data that corresponds to a data slice, wherein the query includes one or more identifiers, wherein the one or more identifiers correspond to one or more data properties included in the data slice;
determining whether a bloom filter includes the one or more identifiers based on the query;
in response to determining that the bloom filter includes the one or more identifiers, determining that the data slice includes the data requested by the query based on the bloom filter; and
in response to determining that the data slice includes the data requested by the query, loading the data slice to a memory.
10 . The computer-implemented method of claim 9 , further comprising:
receiving a second query that corresponds to a second data slice;
determining that no bloom filter is available for the second query; and
in response to determining that no bloom filter is available, loading the second data slice to the memory.
11 . The computer-implemented method of claim 10 , wherein the determining that no bloom filter is available further comprises determining that no bloom filter corresponds to the second data slice.
12 . The computer-implemented method of claim 10 , wherein the determining that no bloom filter is available further comprises:
determining that a second bloom filter corresponds to the data slice;
determining that the second query indicates an identifier; and
determining that the second bloom filter does not correspond to the identifier.
13 . The computer-implemented method of claim 12 , further comprising:
in response to determining that the second bloom filter does not correspond to the identifier, adding 1 to a bloom filter counter of the second bloom filter;
determining that the bloom filter counter is more than a threshold; and
invalidating the bloom filter.
14 . The computer-implemented method of claim 13 , further comprising creating a new bloom filter for the second data slice, wherein the new bloom filter corresponds to at least the identifier.
15 . The computer-implemented method of claim 9 , further comprising:
receiving a further query corresponding to a further data slice;
determining a further bloom filter based on the further query;
determining that the further data slice does not include data requested by the further query based on the further bloom filter; and
in response to determining that the further data slice does not include the data requested by the further query, refraining from loading the further data slice to the memory.
16 . The computer-implemented method of claim 9 , further comprising:
determining a change on the data slice within a predetermined time period; and
invalidating the bloom filter.
17 . A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations, the operations comprising:
receiving a query to retrieve data that corresponds to a data slice, wherein the query includes one or more identifiers, wherein the one or more identifiers correspond to one or more data properties included in the data slice;
determining whether a bloom filter includes the one or more identifiers based on the query;
in response to determining that the bloom filter includes the one or more identifiers, determining that the data slice includes the data requested by the query based on the bloom filter; and
in response to determining that the data slice includes the data requested by the query, loading the data slice to a memory.
18 . The non-transitory computer-readable device of claim 17 , wherein the operations further comprises:
receiving a second query that corresponds to a second data slice;
determining that no bloom filter is available for the second query; and
in response to determining that no bloom filter is available, loading the second data slice to the memory.
19 . The non-transitory computer-readable device of claim 18 , wherein the determining that no bloom filter is available further comprises determining that no bloom filter corresponds to the second data slice.
20 . The non-transitory computer-readable device of claim 18 , wherein the determining that no bloom filter is available further comprises:
determining that a second bloom filter corresponds to the data slice;
determining that the second query indicates an identifier; and
determining that the second bloom filter does not correspond to the identifier.