Combined filtering and indexing for read-only data sets
The present disclosure provides a method for executing a query in a database, the method comprising receiving the query that comprises a query key; searching a data structure of the database to determine whether the query key includes a portion that corresponds to an entry in the database, wherein the data structure comprises a first sequence of values comprising information that indicates whether the database comprises data that corresponds to a portion of a key, and a second sequence of values comprising pointers pointing to data that corresponds to the key; in response to the query key including a portion that corresponds to an entry in the database, determining a location of a value in the first sequence, wherein the value corresponds to the portion of the query key; and determining a pointer in the second sequence of values according to the location, wherein the pointer points to data that corresponds to the query key.
1. A method for executing a query in a database, the method comprising:
receiving the query that comprises a query key;
searching a data structure of the database to determine whether the query key includes a portion that corresponds to an entry in the database, wherein the data structure comprises a first sequence of values comprising information that indicates whether the database comprises data that corresponds to a portion of a key, and a second sequence of values comprising pointers pointing to data that corresponds to the key;
in response to determining the query key including a portion that corresponds to an entry in the first sequence of values of the database, determining a location of a value in the first sequence, wherein the value corresponds to the portion of the query key; and
determining a pointer in the second sequence of values according to the location, wherein the pointer points to data that corresponds to the query key.
2. The method according to claim 1 , further comprising:
retrieving a data run that the pointer points to; and
retrieving, from the data run, data that correspond to the query key.
3. The method according to claim 1 , wherein the first sequence of values and the second sequence of values can be retrieved separately.
4. The method according to claim 1 , wherein the data structure comprises a filter for the database, the filter comprising a succinct trie.
5. The method according to claim 4 , wherein the first sequence of values comprises hash values.
6. The method according to claim 5 , wherein:
the succinct trie is organized according to shared prefixes of keys; and
searching a filter of the database to determine whether the query key includes a prefix that corresponds to an entry in the database further comprising:
searching the succinct trie by following a prefix of the query key;
determining a hash value that corresponds to the prefix of the query key; and
determining whether the prefix of the query key exists in the database according to the hash value.
7. The method according to claim 4 , wherein the succinct trie comprises a Level-Orderd Unary Degree Sequence.
8. A data storage system for executing a query in a database, comprising:
a memory storing a set of instructions; and
a processor configured to execute the set of instructions to cause the data storage system to:
receive the query that comprises a query key;
search a data structure of the database to determine whether the query key includes a portion that corresponds to an entry in the database, wherein the data structure comprises a first sequence of values comprising information that indicates whether the database may comprise data that corresponds to a portion of a key, and a second sequence of values comprising pointers pointing to data that corresponds to the key;
in response to determining the query key including a portion that corresponds to an entry in the first sequence of values of the database, determine a location of a value in the first sequence, wherein the value corresponds to the portion of the query key; and
determine a pointer in the second sequence of values according to the location, wherein the pointer points to data that corresponds to the query key.
9. The data storage system according to claim 8 , wherein the one or more processors are further configured to executed the set of instructions to cause the system to:
retrieving a data run that the pointer points to; and
retrieving, from the data run, data that correspond to the query key.
10. The data storage system according to claim 8 , wherein the first sequence of values and the second sequence of values can be retrieved separately.
11. The data storage system according to claim 8 , wherein the data structure comprises a filter for the database, the filter comprising a succinct trie.
12. The data storage system according to claim 11 , wherein the first sequence of values comprises hash values.
13. The data storage system according to claim 12 , wherein:
the succinct trie is organized according to shared prefixes of keys; and
the one or more processors are further configured to executed the set of instructions to cause the system to:
search the succinct trie by following a prefix of the query key;
determine a hash value that corresponds to the prefix of the query key; and
determine whether the prefix of the query key exists in the database according to the hash value.
14. The data storage system according to claim 11 , wherein the succinct trie comprises a Level-Orderd Unary Degree Sequence.
15. A non-transitory computer readable medium that stores a set of instructions that is executable by one or more processors of an apparatus to cause the apparatus to initiate a method for executing a query in a database, the method comprising:
receiving the query that comprises a query key;
searching a data structure of the database to determine whether the query key includes a portion that corresponds to an entry in the database, wherein the data structure comprises a first sequence of values comprising information that indicates whether the database comprises data that corresponds to a portion of a key, and a second sequence of values comprising pointers pointing to data that corresponds to the key;
in response to determining the query key including a portion that corresponds to an entry in the first sequence of values of the database, determining a location of a value in the first sequence, wherein the value corresponds to the portion of the query key; and
determining a pointer in the second sequence of values according to the location, wherein the pointer points to data that corresponds to the query key.
16. The non-transitory computer readable medium according to claim 15 , wherein the method further comprising:
retrieving a data run that the pointer points to; and
retrieving, from the data run, data that correspond to the query key.
17. The non-transitory computer readable medium according to claim 15 , wherein the first sequence of values and the second sequence of values can be retrieved separately.
18. The non-transitory computer readable medium according to claim 15 , wherein:
the data structure comprises a filter of the database, the filter comprising a succinct trie;
the first sequence of values comprises hash values;
the succinct trie is organized according to shared prefixes of keys; and
the method further comprises:
searching the succinct trie by following a prefix of the query key; determining a hash value that corresponds to the prefix of the query key; and
determining whether the prefix of the query key exists in the database according to the hash value.
19. The non-transitory computer readable medium according to claim 18 , wherein the succinct trie comprises a Level-Orderd Unary Degree Sequence.