System and method for facilitating improved performance in ordering key-value storage with input/output stack simplification
During operation, a key-value storage system can receive a request to write data to a data region in a first non-volatile memory. The system can determine a key associated with the data and the key can correspond to an entry in a data structure maintained by a volatile memory the storage system. In response to determining the key, the system can write the data to the data region in the first non-volatile memory. The system can update in the volatile memory the data structure entry corresponding to the key with a physical location in the data region of the first non-volatile memory to which the data is written. The system can write the data structure update to a second non-volatile memory. The system can then store a snapshot of the data structure in a metadata region associated with the first non-volatile memory, thereby allowing persistent storage of the data structure.
1. A computer-implemented method for facilitating a storage system, the method comprising: initializing a data structure maintained by a volatile memory of the storage system by pre-allocating the data structure in the volatile memory of the storage system with a set of keys in ascending predetermined order and a corresponding set of initialized values; receiving a request to write data to a data region in a first non-volatile memory; determining a key associated with the data, wherein the key corresponds to an entry in a the data structure maintained by a the volatile memory the of the storage system; in response to determining the key, writing the data to the data region in the first non-volatile memory; updating, in the volatile memory, the data structure entry corresponding to the key with a physical location in the data region of the first non-volatile memory to which the data is written; and writing the data structure update to a second non-volatile memory.
2. The method of claim 1 , wherein updating the data structure further comprises: modifying the data structure entry by reconnecting a unique correspondence between the key and the physical location while maintaining the physical location of the data in the data region of the first non-volatile memory.
3. The method of claim 1 , wherein the entry in the data structure indicates: the key associated with the data; and the physical location in the first non-volatile memory at which the data is written, wherein the physical location is represented by a physical offset value and a length of the data.
4. The method of claim 1 , wherein the volatile system memory comprises a dynamic random access memory (DRAM); wherein the first non-volatile memory comprises a Not-and (NAND) flash memory; and wherein the second non-volatile memory represents a persistent journal.
5. The method of claim 1 , wherein writing the data structure update to the second non-volatile memory comprises: sequentially appending the data structure update in the second non-volatile memory.
6. The method of claim 1 , further comprising: storing a snapshot of the data structure in a metadata region in the first non-volatile memory by performing a periodical sequential flushing of the snapshot of the data structure to the metadata region in the first non-volatile memory.
7. The method of claim 1 , further comprising: when the data structure stored in the volatile memory is lost or corrupted, rebuilding the data structure by using a recent snapshot in a metadata region in the first non-volatile memory and a recent data structure update in the second non-volatile memory.
8. A computer system, comprising: a processor; and a memory coupled to the processor and storing instructions, which when executed by the processor cause the processor to perform a method for facilitating a storage system, the method comprising: initializing a data structure maintained by a volatile memory of the storage system by pre-allocating the data structure in the volatile memory of the storage system with a set of keys in ascending predetermined order and a corresponding set of initialized values; receiving a request to write data to a data region in a first non-volatile memory; determining a key associated with the data, wherein the key corresponds to an entry in a the data structure maintained by a the volatile memory the of the storage system; in response to determining the key, writing the data to the data region in the first non-volatile memory; updating, in the volatile system memory, the data structure entry corresponding to the key with a physical location in the data region of the first non-volatile memory to which the data is written; and writing the data structure update to a second non-volatile memory.
9. The computer system of claim 8 , wherein updating the data structure further comprises: modifying the data structure entry by reconnecting a unique correspondence between the key and the physical location while maintaining the physical location of the data in the data region of the first non-volatile memory.
10. The computer system of claim 8 , wherein the entry in the data structure indicates: the key associated with the data; and the physical location in the first non-volatile memory at which the data is written, wherein the physical location is represented by a physical offset value and a length of the data.
11. The computer system of claim 8 , wherein the volatile system memory comprises a dynamic random access memory (DRAM); wherein the first non-volatile memory comprises a Not-and (NAND) flash memory; and wherein the second non-volatile memory represents a persistent journal.
12. The computer system of claim 8 , wherein writing the data structure update to the second non-volatile memory, comprises: sequentially appending the data structure update in the second non-volatile memory.
13. The computer system of claim 8 , wherein the method further comprises: storing a snapshot of the data structure in a metadata region in the first non-volatile memory by performing a periodical sequential flushing of the snapshot of the data structure to the metadata region in the first non-volatile memory.
14. The computer system of claim 8 , further comprising: when the data structure stored in the volatile memory is lost or corrupted, rebuilding the data structure by using a recent snapshot in the metadata region in the first non-volatile memory and a recent data structure update in the second non-volatile memory.
15. An apparatus, comprising: a processor; a controller; a volatile memory; a first non-volatile memory; and a second non-volatile memory; wherein the controller store instructions, which when executed by the processor cause the processor to perform a method for facilitating a storage system, the method comprising: initializing a data structure maintained by a volatile memory of the storage system by pre-allocating the data structure in the volatile memory of the storage system with a set of keys in ascending predetermined order and a corresponding set of initialized values; receiving a request to write data to a data region in a first non-volatile memory; determining a key associated with the data, wherein the key corresponds to an entry in a the data structure maintained by a the volatile memory the of the storage system; in response to determining the key, writing the data to the data region in the first non-volatile memory; updating, in the volatile memory, the data structure entry corresponding to the key with a physical location in the data region of the first non-volatile memory to which the data is written; and writing the data structure update to a second non-volatile memory.
16. The apparatus of claim 15 , wherein writing the data structure update to the second non-volatile memory, comprises: sequentially appending the data structure update in the second non-volatile memory.
17. The apparatus of claim 15 , wherein the method further comprises: when the data structure stored in the volatile memory is lost or corrupted, recovering the data structure by using a recent snapshot in the metadata region in the first non-volatile memory and a recent data structure update in the second non-volatile memory.
18. The apparatus of claim 15 , wherein updating the data structure further comprises: modifying the data structure entry by reconnecting a unique correspondence between the key and the physical location while maintaining the physical location of the data in the data region of the first non-volatile memory.