Key-value store with internal key-value storage interface
View Patent ↗A key-value store is provided having one or more key-value storage interfaces. A key-value store on at least one compute node comprises a memory for storing a plurality of key-value pairs; and an abstract storage interface comprising a software interface module that communicates with at least one persistent storage device providing a key-value interface for persistent storage of one or more of the plurality of key-value pairs, wherein the software interface module provides the one or more key-value pairs to the at least one persistent storage device in a key-value format. The abstract storage interface optionally processes one or more batch operations on the plurality of key-value pairs. A distributed embodiment for a partitioned key-value store is also provided.
1. A key-value store on at least one compute node, comprising:
a memory for storing a plurality of key-value pairs, wherein keys in said key-value pairs comprise semantic information; and
an abstract storage interface comprising a software interface module that communicates with at least one persistent storage device providing a key-value interface for persistent storage of one or more of said plurality of key-value pairs, wherein said software interface module provides said one or more key-value pairs to said at least one persistent storage device in a key-value format that maintains said semantic information of said keys in said key-value pairs.
2. The key-value store of claim 1 , wherein said at least one persistent storage device comprises one or more of a flash storage device providing a key-value interface and a disk storage device providing a key-value interface.
3. The key-value store of claim 2 , wherein said flash storage device providing said key-value interface and said disk storage device providing said key-value interface comprise respective disjoint subsets of storage.
4. The key-value store of claim 1 , wherein said abstract storage interface processes one or more batch operations on said plurality of key-value pairs.
5. The key-value store of claim 1 , wherein said abstract storage interface provides a storage-specific interface for a plurality of storage destination types.
6. The key-value store of claim 1 , wherein said abstract storage interface provides a key-value interface between one or more local key-value stores and one or more additional storage devices having a key-value interface.
7. The key-value store of claim 1 , wherein a processor in said at least one persistent storage device performs one or more of index building, reorganization, resorting, deduplication and compression of said one or more key-value pairs.
8. A distributed key-value store, comprising:
a plurality of compute nodes connected by a network, wherein at least one of said compute nodes comprises a partitioned key-value store server;
wherein the partitioned key-value store server is configured to store key-value data using a local key-value store, wherein keys in said key-value data comprise semantic information, said local key-value store comprising an abstract storage interface comprising a software interface module that communicates with at least one persistent storage device providing a key-value interface for persistent storage of one or more key-value pairs, wherein said software interface module provides said one or more key-value pairs to said at least one persistent storage device in a key-value format that maintains said semantic information of said keys in said key-value pairs.
9. The distributed key-value store of claim 8 , wherein the partitioned key-value store server is configured to communicate with a plurality of partitioned key-value store clients over a network and to process requests from said plurality of partitioned key-value store clients.
10. The distributed key-value store of claim 8 , wherein the partitioned key-value store server is further configured to communicate with one or more additional partitioned key-value store servers over a network and to process requests from said one or more additional partitioned key-value store servers.
11. The distributed key-value store of claim 8 , further comprising at least one additional partitioned key-value store server, wherein each partitioned key-value store server stores a globally sorted range of keys.
12. The distributed key-value store of claim 8 , further comprising at least one additional partitioned key-value store server, wherein each partitioned key-value store server stores a locally sorted range of keys within a set of string table files (SST).
13. The distributed key-value store of claim 8 , further comprising at least one additional partitioned key-value store server, wherein at least one of said partitioned key-value store servers processes one or more batch operations on one or more sorted ranges of keys.
14. A method comprising:
providing a key-value store server on at least one compute node of a distributed storage system for storing a plurality of key-value pairs, wherein keys in said key-value pairs comprise semantic information; and
communicating with at least one persistent storage device providing a key-value interface for persistent storage of one or more of said plurality of key-value pairs using an abstract storage interface comprising a software interface module, wherein said software interface module provides said one or more key-value pairs to said at least one persistent storage device in a key-value format that maintains said semantic information of said keys in said key-value pairs.
15. The method of claim 14 , wherein said at least one persistent storage device comprises one or more of a flash storage device providing a key-value interface and a disk storage device providing a key-value interface.
16. The method of claim 14 , wherein said abstract storage interface processes one or more batch operations on said plurality of key-value pairs.
17. The method of claim 14 , wherein said abstract storage interface provides a storage-specific interface for a plurality of storage destination types.
18. The method of claim 14 , wherein said abstract storage interface provides a key-value interface between one or more local key-value stores and one or more additional storage devices having a key-value interface.
19. The method of claim 14 , wherein a processor in said at least one persistent storage device performs one or more of index building, reorganization, resorting, deduplication and compression of said one or more key-value pairs.
20. The method of claim 14 , wherein said key-value store server comprises a partitioned key-value store server configured to communicate with one or more additional partitioned key-value store servers over a network and to process requests from said one or more additional partitioned key-value store servers.
21. The key-value store of claim 6 , wherein said one or more local key-value stores comprise a different hardware tier comprising respectively disjoint subsets of storage relative to said one or more additional storage devices.
22. The method of claim 18 , wherein said one or more local key-value stores comprise a different hardware tier comprising respectively disjoint subsets of storage relative to said one or more additional storage devices.