Parallel storage system with burst buffer appliance for storage of partitioned key-value store across a plurality of storage tiers
View Patent ↗Parallel storage systems are provided with a burst buffer appliance for storage of a partitioned key-value store across a plurality of storage tiers. The burst buffer appliance is configured to store key-value data on first and second storage tiers comprising respective disjoint subsets of storage. One or more servers in the partitioned key-value store provide (i) an interface between one or more local key-value stores and one or more additional storage architectures, and/or (ii) a key value interface that provides access to at least one hardware key-value store. The local key-values are optionally sorted both locally by the local store and globally by the overall MDHIM framework. The local and global sorting can be leveraged to provide batch input/output (IO) operations that aggregate multiple requests into batch operations on one or more sorted ranges.
1. A distributed storage system for storing a partitioned key-value store, comprising:
a first storage tier comprising persistent storage of a first persistent storage type;
a second storage tier comprising persistent storage of a second persistent storage type different than the first persistent storage type, wherein the first storage tier comprises a high-speed memory having a substantially lower access time than the second storage tier;
a plurality of partitioned key-value store servers that store a corresponding plurality of partitions of the partitioned key-value store on one or more of said first storage tier and said second storage tier; and
a burst buffer appliance comprising a high-speed memory having a substantially lower access time than the second storage tier and software to access said first storage tier and said second storage tier, wherein said first storage tier is one or more of part of said burst buffer appliance and external to said burst buffer appliance,
wherein the burst buffer appliance communicates with at least one compute node over a network, wherein the burst buffer appliance is coupled between the at least one compute node and the partitioned key-value store maintained on one or more of said first storage tier and said second storage tier,
the software of the burst buffer appliance processes one or more tiering commands to store key-value data of the partitioned key-value store on said first and second storage tiers comprising respective disjoint subsets of storage,
wherein said partitioned key-value store comprises the plurality of partitions stored by the corresponding plurality of the partitioned key-value store servers, wherein each partition comprises a distinct range of keys, and wherein a range-knowledgeable client executing in said distributed storage system obtains range information from said corresponding plurality of partitioned key-value store servers in response to a global command, wherein said range information indicates that a given partitioned key-value store server stores a given one of said distinct range of keys, and wherein said range-knowledgeable client sends one or more requests for key-value data within said given distinct range of keys directly to said given partitioned key-value store server.
2. The distributed storage system of claim 1 , wherein said burst buffer appliance processes said one or more tiering commands from said given partitioned key-value store server on a given compute node of the at least one compute node in said distributed storage system.
3. The distributed storage system of claim 1 , wherein said burst buffer appliance processes one or more of read and write commands from a local key value store on a given compute node of the at least one compute node in said distributed storage system.
4. The distributed storage system of claim 1 , wherein said burst buffer appliance comprises a partitioned key-value store server.
5. The distributed storage system of claim 1 wherein the burst buffer appliance communicates with a plurality of partitioned key-value store clients over a network and to process requests from said plurality of partitioned key-value store clients.
6. The distributed storage system of claim 1 , wherein each partitioned key-value store server stores a globally sorted range of keys in said partitioned key-value store.
7. The distributed storage system of claim 6 , wherein each partitioned key-value store server stores a locally sorted range of keys.
8. The distributed storage system of claim 1 , wherein at least one partitioned key-value store server provides an interface between one or more local key-value stores and one of a plurality of additional storage architectures.
9. The distributed storage system of claim 8 , wherein said plurality of additional storage architectures comprise one or more of a burst buffer appliance, a Simple Storage Service (S3) object store and a Portable Operating System Interface (POSIX).
10. The distributed storage system of claim 1 , wherein at least one partitioned key-value store server provides a key value interface that provides access to at least one hardware key-value store.
11. The distributed storage system of claim 1 , wherein at least one partitioned key-value store server processes one or more batch operations on one or more sorted ranges of said partitioned key-value store.
12. A method comprising:
communicating, by a burst buffer appliance in a distributed storage system, with a plurality of compute nodes over a network, a first storage tier comprising persistent storage of a first persistent storage type, and a second storage tier comprising persistent storage of a second persistent storage type different than the first persistent storage type, wherein the first storage tier comprises a high-speed memory having a substantially lower access time than the second storage tier;
storing a plurality of partitions of a partitioned key-value store on one or more of said first storage tier and said second storage tier using a corresponding plurality of partitioned key-value store servers; and
processing, by the burst buffer appliance, one or more tiering commands to store key-value data of the partitioned key-value store on said first and second storage tiers comprising respective disjoint subsets of storage,
the burst buffer appliance comprising a high-speed memory having a substantially lower access time than the second storage tier and software to access said first storage tier and said second storage tier, wherein the burst buffer appliance is coupled between the plurality of compute nodes and the partitioned key-value store maintained on one or more of said first storage tier and said second storage tier,
wherein said partitioned key-value store comprises the plurality of partitions stored by the corresponding plurality of partitioned key-value store servers, wherein each partition comprises a distinct range of keys, and wherein a range-knowledgeable client executing in said distributed storage system obtains range information from said corresponding plurality of partitioned key-value store servers in response to a global command, wherein said range information indicates that a given partitioned key-value store server stores a given one of said distinct range of keys, and wherein said range-knowledgeable client sends one or more requests for key-value data within said given distinct range of keys directly to said given partitioned key-value store server.
13. The method of claim 12 , further comprising said burst buffer appliance processing said one or more tiering commands from said given partitioned key-value store server on a compute node in said distributed storage system.
14. The method of claim 12 , further comprising said burst buffer appliance processing one or more of read and write commands from a local key value store on a compute node in said distributed storage system.
15. The method of claim 12 , wherein said burst buffer appliance comprises a partitioned key-value store server.
16. The method of claim 12 , wherein the plurality of partitioned key-value store servers store a globally sorted range of keys in a respective partitioned key-value store.
17. The method of claim 16 , wherein the plurality of partitioned key-value store servers store a locally sorted range of keys.
18. The method of claim 12 , wherein at least one of the plurality of partitioned key-value store servers provides an interface between one or more local key-value stores and one of a plurality of additional storage architectures.
19. The method of claim 18 , wherein said plurality of additional storage architectures comprise one or more of a burst buffer appliance, a Simple Storage Service (S3) object store and a Portable Operating System Interface (POSIX).
20. The method of claim 12 , wherein at least one of the plurality of partitioned key-value store servers provides a key value interface that provides access to at least one hardware key-value store.
21. The method of claim 12 , wherein at least one of the plurality of partitioned key-value store servers processes one or more batch operations on one or more sorted ranges of said partitioned key-value store.
22. A partitioned key-value store comprising:
a first storage tier comprising persistent storage of a first persistent storage type;
a second storage tier comprising persistent storage of a second persistent storage type different than the first persistent storage type, wherein the first storage tier comprises a high-speed memory having a substantially lower access time than the second storage tier; and
a burst buffer appliance comprising a high-speed memory having a substantially lower access time than the second storage tier and software to access said first storage tier and said second storage tier; and
a plurality of partitioned key-value store servers coupled to the burst buffer appliance via a network, wherein the plurality of partitioned key-value store servers store a corresponding plurality of partitions of the partitioned key-value store on one or more of said first storage tier and said second storage tier;
wherein the burst buffer appliance communicates with at least one compute node over the network, wherein the burst buffer appliance is coupled between the at least one compute node and the partitioned key-value store maintained on one or more of said first storage tier and said second storage tier,
the software of the burst buffer appliance processes one or more tiering commands to store key-value data of the partitioned key-value store on said first and second storage tiers comprising respective disjoint subsets of storage,
wherein said partitioned key-value store comprises the plurality of partitions stored by the corresponding plurality of the partitioned key-value store servers, wherein each partition comprises a distinct range of keys, and wherein a range-knowledgeable client of said partitioned key-value store obtains range information from said corresponding plurality of partitioned key-value store servers in response to a global command, wherein said range information indicates that a given partitioned key-value store server stores a given one of said distinct range of keys, and wherein said range-knowledgeable client sends one or more requests for key-value data within said given distinct range of keys directly to said given partitioned key-value store server.
23. The distributed storage system of claim 1 , wherein the high speed memory of said burst buffer appliance comprises the first storage tier.
24. The distributed storage system of claim 1 , wherein the high speed memory of said burst buffer appliance is used to access the first storage tier and the second storage tier.