Balanced key range based retrieval of key-value database
A data object is received for storage in a key-value store. A partitioning token prefix is generated for the data object. A logical key for the data object is determined. A partitioning key is generated based at least in part on combining the partitioning token prefix and the logical key. Data associated with the data object is stored in the key-value store based on the partitioning key.
1. A system, comprising:
a processor configured to:
receive a first data object associated with a first label path for storage in a key-value store;
generate a first partitioning token prefix for the first data object;
determine a first logical key comprising the first label path for the first data object;
generate a first partitioning key comprising the first partitioning token prefix and the first logical key;
receive a second data object associated with a second label path for storage in the key-value store;
generate a second partitioning token prefix for the second data object;
determine a second logical key comprising the second label path for the second data object;
generate a second partitioning key comprising the second partitioning token prefix and the second logical key; and
execute load balanced storage by:
storing a first data value associated with the first data object in a first node of the key-value store based on the first partitioning token prefix in the first partitioning key; and
storing a second data value associated with the second data object in a second node of the key-value store based on the second partitioning token prefix in the second partitioning key; and
a memory coupled to the processor and configured to provide the processor with instructions.
2. The system recited in claim 1 , wherein the first partitioning token prefix for the first data object is based at least in part on a randomizing hash.
3. The system recited in claim 1 , wherein the first logical key is determined based at least in part on an ordinal range associated with the first data object.
4. The system recited in claim 1 , wherein the processor is further configured to:
receive a request to find data based on a range condition against the first logical key;
create a partitioning key range for the first partitioning token prefix; and
submit a range query to the key-value store based at least in part on the partitioning key range.
5. The system recited in claim 1 , wherein the first partitioning token prefix is identical for all columnar blocks of the same objects.
6. The system recited in claim 1 , wherein the first partitioning token prefix is a combination of a hash of a primary identifier associated with the first data object and a hash of a collection name associated with the first data object.
7. The system recited in claim 1 , wherein the first logical key further comprises an ordering chunk associated with the first data object.
8. The system recited in claim 7 , wherein the ordering chunk is a time chunk based on a timestamp associated with the first data object.
9. The system recited in claim 7 , wherein the ordering chunk separates data in a collection into a set of time series.
10. The system recited in claim 7 , wherein the ordering chunk is an age chunk.
11. The system recited in claim 1 , wherein the processor is further configured to determine a portioning key associated with a second level ordered key-value map, wherein the portioning key comprises at least one of: a block key, a column key, a clustering key, an order remainder, and an object id.
12. A method, comprising:
receiving a first data object associated with a first label path for storage in a key-value store;
generating a first partitioning token prefix for the first data object;
determining a first logical key comprising the first label path for the first data object;
generating a first partitioning key comprising the first partitioning token prefix and the first logical key;
receiving a second data object associated with a second label path for storage in the key-value store;
generating a second partitioning token prefix for the second data object;
determining a second logical key comprising the second label path for the second data object;
generating a second partitioning key comprising the second partitioning token prefix and the second logical key; and
executing load balanced storage by:
storing a first data value associated with the first data object in a first node of the key-value store based on the first partitioning token prefix in the first partitioning key; and
storing a second data value associated with the second data object in a second node of the key-value store based on the second partitioning token prefix in the second partitioning key.
13. The method of claim 12 , further comprising:
receiving a request to find data based on a range condition against the first logical key;
creating a partitioning key range for the first partitioning token prefix; and
submitting a range query to the key-value store based at least in part on the partitioning key range.
14. A computer program product, the computer program product being embodied in a tangible computer readable storage medium and comprising computer instructions for:
receiving a first data object associated with a first label path for storage in a key-value store;
generating a first partitioning token prefix for the first data object;
determining a first logical key comprising the first label path for the first data object;
generating a first partitioning key comprising the first partitioning token prefix and the first logical key;
receiving a second data object associated with a second label path for storage in the key-value store;
generating a second partitioning token prefix for the second data object;
determining a second logical key comprising the second label path for the second data object;
generating a second partitioning key comprising the second partitioning token prefix and the second logical key; and
executing load balanced storage by:
storing a first data value associated with the first data object in a first node of the key-value store based on the first partitioning token prefix in the first partitioning key; and
storing a second data value associated with the second data object in a second node of the key-value store based on the second partitioning token prefix in the second partitioning key.
15. The computer program product of claim 14 , further comprising computer instructions for:
receiving a request to find data based on a range condition against the first logical key;
creating a partitioning key range for the first partitioning token prefix; and
submitting a range query to the key-value store based at least in part on the partitioning key range.
16. The system recited in claim 7 , wherein the first data object and the second data object are associated with a collection, wherein the first logical key further comprises a name of the collection, and wherein the second logical key further comprises the name of the collection.
17. The method of claim 12 , wherein the first logical key further comprises an ordering chunk associated with the first data object.
18. The method of claim 17 , wherein the first data object and the second data object are associated with a collection, wherein the first logical key further comprises a name of the collection, and wherein the second logical key further comprises the name of the collection.
19. The computer program product of claim 14 , wherein the first logical key further comprises an ordering chunk associated with the first data object.
20. The computer program product of claim 19 , wherein the first data object and the second data object are associated with a collection, wherein the first logical key further comprises a name of the collection, and wherein the second logical key further comprises the name of the collection.