Probabilistic data structures embedded in database indexes
Embedded probabilistic data structures may be maintained in database indexes. Different data pages linked in a database index structure may have different respective probabilistic data structures stored within the index. When an access request for the database is received, the probabilistic data structure for a particular database page that is identified as possibly storing an item for the access request may be evaluated to determine whether or not to obtain the data page. If the probabilistic data structure indicates the possible presence of the item, then the data page is obtained to perform the request. If the probabilistic data structure indicates that the item is not present, then the data page is not obtained.
1 . A system, comprising:
at least one processor; and
a memory, storing program instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to implement a database system, configured to:
receiving a request to access an item;
perform the request to access the item, wherein to perform the request, the database system is configured to:
traverse an index structure comprising a plurality of index nodes, wherein the traversal identifies both:
an index node that references a data page to obtain to perform the request, wherein the data page is one of a plurality of data pages that store database data; and
a probabilistic data structure maintained for the data page within the index node out of a plurality of probabilistic data structures that are separately stored for the plurality of data pages within different ones of the plurality of index nodes;
evaluate the probabilistic data structure maintained for the data page within the index node to determine that the data page should not be obtained as part of performing the request, wherein the probabilistic data structure indicates whether a given item is possibly stored in the data page or not stored in the data page.
2 . The system of claim 1 , wherein the database system is further configured to:
receive a different request to access a different item;
perform the different request to access the different item, wherein to perform the request, the database system is configured to:
traverse the index structure that includes the index node to identify the data page to obtain to perform the different request;
evaluate the probabilistic data structure maintained for the data page to determine that the data page should be obtained as part of performing the different request; and
obtain the data page to perform the different request to access the different item.
3 . The system of claim 1 , wherein the database system is further configured to update the probabilistic data structure for the data page according to a change modifying which items are present in the data page.
4 . The system of claim 1 , wherein the database system is further configured to generate a new probabilistic data structure in the database data as part of compacting one or more data object files stored as part of a backup of the data base.
5 . A method, comprising:
receiving a request to access an item at a database system;
performing, by the database system, the request to access the item, comprising:
evaluating, by the database system, an index structure comprising a plurality of index nodes, wherein the evaluating identifies both:
an index node for a data page to obtain to perform the request, wherein the data page is one of a plurality of data pages that store database data; and
a probabilistic data structure maintained for the data page within the index node out of a plurality of probabilistic data structures that are separately stored for the plurality of data pages within different ones of the plurality of index nodes;
evaluating, by the database system, the probabilistic data structure maintained for the data page within the index node to determine that the data page should not be obtained as part of performing the request, wherein the probabilistic data structure indicates whether a given item is possibly stored in the data page or not stored in the data page.
6 . The method of claim 5 , further comprising:
receiving a different request to access a different item at the database system;
performing, by the database system, the different request to access the different item, comprising:
evaluating, by the database system, the index structure to identify the data page to obtain to perform the different request;
evaluating, by the database system, the probabilistic data structure maintained for the data page to determine that the data page should be obtained as part of performing the different request; and
obtaining, by the database system, the data page to perform the different request to access the different item.
7 . The method of claim 5 , wherein the probabilistic data structure is a bloom filter.
8 . The method of claim 5 , wherein the index structure is a b+tree.
9 . The method of claim 5 , further comprising updating the probabilistic data structure for the data page according to a change modifying which items are present in the data page.
10 . The method of claim 5 , further comprising generating a new probabilistic data structure in the database data as part of compacting one or more data object files stored as part of a backup of the data base.
11 . The method of claim 5 , wherein the index structure is stored as part of a data object generated from a change log for the database data that is stored as part of a backup of the database data and wherein the database data is a partition of a database.
12 . The method of claim 5 , wherein the database system is implemented as part of a non-relational database service offered by a provider network.
13 . One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to implement:
receiving a request to access an item at a database system;
performing, by the database system, the request to access the item, comprising:
evaluating, by the database system, an index structure comprising a plurality of index nodes, wherein the evaluating identifies both:
an index node for a data page to obtain to perform the request, wherein the data page is one of a plurality of data pages that store database data; and
a probabilistic data structure maintained for the data page within the index node out of a plurality of probabilistic data structures that are separately stored for the plurality of data pages within different ones of the plurality of index nodes; and
evaluating, by the database system, probabilistic data structure maintained for the data page within the index node to determine that the data page should not be obtained as part of performing the request, wherein the probabilistic data structure indicates whether a given item is possibly stored in the data page or not stored in the data page.
14 . The one or more non-transitory, computer-readable storage media of claim 13 , storing further instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement:
receiving a different request to access a different item at the database system;
performing, by the database system, the different request to access the different item, comprising:
evaluating, by the database system, the index structure to identify the data page to obtain to perform the different request;
evaluating, by the database system, the probabilistic data structure maintained for the data page to determine that the data page should be obtained as part of performing the different request; and
obtaining, by the database system, the data page to perform the different request to access the different item.
15 . The one or more non-transitory, computer-readable storage media of claim 13 , wherein the probabilistic data structure is a bloom filter.
16 . The one or more non-transitory, computer-readable storage media of claim 13 , storing further instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement updating the probabilistic data structure for the data page according to a change modifying which items are present in the data page.
17 . The one or more non-transitory, computer-readable storage media of claim 13 , wherein the index structure is a b+tree.
18 . The one or more non-transitory, computer-readable storage media of claim 13 , storing further instructions that when executed on or across the one or more computing devices, cause the one or more computing devices to further implement generating a new probabilistic data structure in the database data as part of compacting one or more data object files stored as part of a backup of the data base.
19 . The one or more non-transitory, computer-readable storage media of claim 13 , wherein the index structure is stored as part of a data object generated from a change log for the database data that is stored as part of a backup of the database data and wherein the database data is a partition of a database.
20 . The one or more non-transitory, computer-readable storage media of claim 13 , wherein the database system is implemented as part of a relational database service offered by a provider network.