Key-value compaction
One embodiment provides a storage device. The storage device includes a storage I/O (input/output) logic and a storage device controller. The storage I/O logic is to couple the storage device to a host device, the storage I/O logic to receive a sort-merge command the host device. The a storage device controller is to identify a level N SSTable (sorted string table) file, a corresponding level N index file, a first level N+1 SSTable file and a corresponding first level N+1 index file, in response to the sort-merge command to be received from the host device. The storage device controller is further to perform a sort-merge of the level N SSTable file and the first level N+1 SSTable file to produce a first level N+1 output SSTable file and a first level N+1 output SSTable index file. The level N SSTable file includes at least one level N key-value (KV) pair. The level N+1 SSTable file includes at least one level N+1 key-value (KV) pair. The sort-merge command includes a level N SSTable file index, a value corresponding to a number of SSTable files included in level N+1 and a level N+1 start SSTable file index. The identifying is based, at least in part, on the level N SSTable file index and the level N+1 start SSTable file index.
1. A storage device comprising:
a storage I/O (input/output) logic to couple the storage device to a host device, the storage I/O logic to receive a sort-merge command the host device; and
a storage device controller to identify a level N SSTable (sorted string table) file, a corresponding level N index file, a first level N+1 SSTable file and a corresponding first level N+1 index file, in response to the sort-merge command to be received from the host device; and to perform a sort-merge of the level N SSTable file and the first level N+1 SSTable file to produce a first level N+1 output SSTable file and a first level N+1 output SSTable index file, wherein the first level N+1 output SSTable file replaces the N SSTable file or replaces the first level N+1 SSTable file,
the level N SSTable file comprising at least one level N key-value (KV) pair, the level N+1 SSTable file comprising at least one level N+1 key-value (KV) pair, the sort-merge command comprising a level N SSTable file index, a value corresponding to a number of SSTable files included in level N+1 and a level N+1 start SSTable file index, the identifying based, at least in part, on the level N SSTable file index and the level N+1 start SSTable file index.
2. The storage device of claim 1 , wherein the storage device controller is to identify a second level N+1 SSTable file and a corresponding second level N+1 index file based, at least in part, on the level N+1 start SSTable file index, and the storage device controller is to perform a sort-merge of the level N SSTable file and the second level N+1 SSTable file to produce a second level N+1 output SSTable file and a second level N+1 output SSTable index file.
3. The storage device of claim 1 , wherein the storage device controller is to at least one of access and/or update an indirection table, the indirection table comprising an array of SSTable file information array elements, each array element corresponding to a respective SSTable file.
4. The storage device of claim 1 , wherein performing the sort-merge comprises comparing a level N key from the level N key-value pair to a level N+1 key from the level N+1 key-value pair and selecting the level N key-value pair if the level N key is less than or equal to the level N+1 key or selecting the level N+1 key-value pair if the level N key is greater than the level N+1 key.
5. The storage device of claim 3 , wherein each array element comprises an SSTable file index, an SSTable file start logical block address (LBA), an SSTable file LBA length, an SSTable index file start LBA, an SSTable index file LBA length, a number of key-value (KV) pairs included in the respective SSTable file and an SSTable file status.
6. The storage device of claim 1 , wherein the storage device controller comprises a sort-merge circuitry comprising at least one key-value compaction architecture.
7. The storage device of claim 1 , wherein the sort-merge command comprises at least one of a command identifier field and/or a field to indicate a number of SSTable files in level N+1 overlapped by the level N SSTable file.
8. A method comprising:
coupling, by a storage I/O (input/output) logic, a storage device to a host device, the storage I/O logic to receive a sort-merge command the host device;
identifying, by a storage device controller, a level N SSTable (sorted string table) file, a corresponding level N index file, a first level N+1 SSTable file and a corresponding first level N+1 index file, in response to the sort-merge command to be received from the host device; and
performing, by the storage device controller, a sort-merge of the level N SSTable file and the first level N+1 SSTable file to produce a first level N+1 output SSTable file and a first level N+1 output SSTable index file, wherein the first level N+1 output SSTable file replaces the N SSTable file or replaces the first level N+1 SSTable file,
the level N SSTable file comprising at least one level N key-value (KV) pair, the level N+1 SSTable file comprising at least one level N+1 key-value (KV) pair, the sort-merge command comprising a level N SSTable file index, a value corresponding to a number of SSTable files included in level N+1 and a level N+1 start SSTable file index, the identifying based, at least in part, on the level N SSTable file index and the level N+1 start SSTable file index.
9. The method of claim 8 , further comprising identifying, by the storage device controller, a second level N+1 SSTable file and a corresponding second level N+1 index file based, at least in part, on the level N+1 start SSTable file index, and performing, by the sort-merge circuitry, a sort-merge of the level N SSTable file and the second level N+1 SSTable file to produce a second level N+1 output SSTable file and a second level N+1 output SSTable index file.
10. The method of claim 8 , further comprising at least one of accessing and/or updating, by the storage device controller, an indirection table, the indirection table comprising an array of SSTable file information array elements, each array element corresponding to a respective SSTable file.
11. The method of claim 8 , wherein performing the sort-merge comprises comparing a level N key from the level N key-value pair to a level N+1 key from the level N+1 key-value pair and selecting the level N key-value pair if the level N key is less than or equal to the level N+1 key or selecting the level N+1 key-value pair if the level N key is greater than the level N+1 key.
12. The method of claim 10 , wherein each array element comprises an SSTable file index, an SSTable file start logical block address (LBA), an SSTable file LBA length, an SSTable index file start LBA, an SSTable index file LBA length, a number of key-value (KV) pairs included in the respective SSTable file and an SSTable file status.
13. The method of claim 8 , wherein the storage device controller comprises a sort-merge circuitry comprising at least one key-value compaction architecture.
14. The method of claim 8 , wherein the sort-merge command comprises at least one of a command identifier field and/or a field to indicate a number of SSTable files in level N+1 overlapped by the level N SSTable file.
15. A system comprising a storage device comprising:
a plurality of nonvolatile media;
a storage I/O (input/output) logic to couple the storage device to a host device, the storage I/O logic to receive a sort-merge command the host device; and
a storage device controller to identify a level N SSTable (sorted string table) file, a corresponding level N index file, a first level N+1 SSTable file and a corresponding first level N+1 index file, in response to the sort-merge command to be received from the host device; and to perform a sort-merge of the level N SSTable file and the first level N+1 SSTable file to produce a first level N+1 output SSTable file and a first level N+1 output SSTable index file, wherein the first level N+1 output SSTable file replaces the N SSTable file or replaces the first level N+1 SSTable file,
the level N SSTable file comprising at least one level N key-value (KV) pair, the level N+1 SSTable file comprising at least one level N+1 key-value (KV) pair, the sort-merge command comprising a level N SSTable file index, a value corresponding to a number of SSTable files included in level N+1 and a level N+1 start SSTable file index, the identifying based, at least in part, on the level N SSTable file index and the level N+1 start SSTable file index.
16. The system of claim 15 , wherein the storage device controller is to identify a second level N+1 SSTable file and a corresponding second level N+1 index file based, at least in part, on the level N+1 start SSTable file index, and the storage device controller is to perform a sort-merge of the level N SSTable file and the second level N+1 SSTable file to produce a second level N+1 output SSTable file and a second level N+1 output SSTable index file.
17. The system of claim 15 , wherein the storage device controller is to at least one of access and/or update an indirection table, the indirection table comprising an array of SSTable file information array elements, each array element corresponding to a respective SSTable file.
18. The system of claim 15 , wherein performing the sort-merge comprises comparing a level N key from the level N key-value pair to a level N+1 key from the level N+1 key-value pair and selecting the level N key-value pair if the level N key is less than or equal to the level N+1 key or selecting the level N+1 key-value pair if the level N key is greater than the level N+1 key.
19. The system of claim 17 , wherein each array element comprises an SSTable file index, an SSTable file start logical block address (LBA), an SSTable file LBA length, an SSTable index file start LBA, an SSTable index file LBA length, a number of key-value (KV) pairs included in the respective SSTable file and an SSTable file status.
20. The system of claim 15 , wherein the storage device controller comprises a sort-merge circuitry comprising at least one key-value compaction architecture.
21. The system of claim 15 , wherein the sort-merge command comprises at least one of a command identifier field and/or a field to indicate a number of SSTable files in level N+1 overlapped by the level N SSTable file.
22. The system of claim 15 , wherein the storage device controller comprises a sort-merge circuitry comprising a command buffer to store the sort-merge command, a plurality of input buffers to store selected SS table files and selected SS table file indexes, a comparator to compare selected keys, a multiplexer, a plurality of intermediate buffers and an output buffer to store each level N+1 output SSTable file.
23. The system of claim 15 , wherein the storage device is selected from the group comprising a solid-state drive (SSD), a hard disk drive (HDD), a network attached storage (NAS) system, a storage area network (SAN) and/or a redundant array of independent disks (RAID) system.
24. The system of claim 15 , wherein each of the plurality of nonvolatile media is selected from the group comprising a NAND flash memory, a NOR memory, a solid state memory, byte addressable nonvolatile memory devices, ferroelectric memory, silicon-oxide-nitride-oxide-silicon (SONOS) memory, polymer memory, byte addressable random accessible three dimensional crosspoint memory, ferroelectric transistor random access memory, magnetoresistive random access memory, phase change memory, resistive memory, ferroelectric memory, spin-transfer torque memory, thermal assisted switching memory (TAS), millipede memory, floating junction gate memory (FJG RAM), magnetic tunnel junction (MTJ) memory, electrochemical cells (ECM) memory, binary oxide filament cell memory, interfacial switching memory, battery-backed RAM, ovonic memory, nanowire memory and/or electrically erasable programmable read-only memory (EEPROM).