Methods to configure and access scalable object stores using KV-SSDs and hybrid backend storage tiers of KV-SSDs, NVMe-SSDs and other flash devices
A system is disclosed. The system may include a computer system, which may include a processor that may execute instructions of an application that accesses an object using an object command, and a memory storing the instructions of the application. The computer system may also include a conversion module to convert the object command to a key-value (KV) command. Finally, the system may include a storage device storing data for the object and processing the object using the KV command.
1. A system, comprising:
a processor executing an instruction accessing a data using a first command;
a processing module to generate a second command based on the first command and insert the second command in a memory-based fabric command; and
at least one storage device storing the data and processing the data using the second command.
2. The system according to claim 1 , wherein:
the data includes an object;
the second command includes a key-value (KV) command;
the at least one storage device storing the data for the object includes a KV-Solid State Drive (SSD); and
the system is configured to transmit the KV command to the KV-SSD.
3. The system according to claim 1 , wherein:
the at least one storage device storing the data includes a Non-Volatile Memory Express (NVMe) SSD;
the processing module includes a translator to generate an NVMe command using the second command, a Logical Block Address (LBA) and an offset for the data; and
the system is configured to transmit the NVMe command to the NVMe SSD.
4. The system according to claim 1 , wherein:
the at least one storage device storing the data further includes a KV-SSD storing a metadata for the data; and
the processing module includes a translator to access an LBA and an offset for the data from the metadata for the data from the KV-SSD.
5. The system according to claim 1 , the system further including:
a second system;
a first transmitter to transmit the fabric command to the second system; and
a first receiver to receive the data from the second system, wherein the second system includes:
the at least one storage device;
a second receiver to receive the fabric command;
a second processing module to recover the second command from the fabric command; and
a second transmitter to transmit the data to the system.
6. The system according to claim 5 , wherein:
the data includes an object;
the second command includes a key-value (KV) command;
the at least one storage device storing the data for the object includes a KV-SSD; and
the second system is configured to transmit the KV command to the KV-SSD.
7. The system according to claim 5 , wherein:
the at least one storage device storing the data includes an NVMe SSD;
the system further includes a translator to generate an NVMe command using the second command, an LBA and an offset for the data; and
the second system is configured to transmit the NVMe command to the NVMe SSD.
8. The system according to claim 5 , the system further comprising:
a KV-SSD storing metadata for the data, the metadata for the data including an LBA and an offset for the data;
a translator configured to access the LBA and the offset for the data from the metadata for the data from the KV-SSD; and
an association module to insert an NVMe command, including the LBA and the offset for the data, in the fabric command, wherein the second system includes a recovery module to recover the NVMe command from the fabric command.
9. A method, comprising:
receiving a first command from a processor, the first command relating to a data;
generating a second command based on the first command;
inserting the second command in a memory-based fabric command; and
sending the fabric command to a storage device storing the data.
10. A method according to claim 9 , wherein:
the storage device stores the data associated with a key; and
generating the second command includes hashing a data identifier (ID) for the data to generate a key.
11. A method according to claim 9 , wherein sending the fabric command to a storage device storing the data includes sending the fabric command to a key-value (KV) Solid State Drive (SSD) in a system remote from the processor, the KV-SSD storing the data in an object.
12. A method according to claim 9 , wherein sending the fabric command to a storage device storing the data further includes:
recovering the second command from the fabric command on the system remote from the processor; and
executing the second command on the storage device in the system remote from the processor.
13. A method according to claim 9 , wherein sending the fabric command to a storage device storing the data includes sending the fabric command to a Non-Volatile Memory Express (NVMe) SSD in a system remote from the processor, the NVMe SSD storing the data.
14. A method according to claim 13 , wherein sending the fabric command to a storage device storing the data further includes:
recovering the second command from the fabric command on the system remote from the processor;
constructing an NVMe command from the second command; and
executing the NVMe command on the NVMe SSD in the system remote from the processor.
15. A method according to claim 9 , wherein associating the second command with a fabric command includes:
sending the second command to a KV-SSD local to the processor, the KV-SSD storing metadata for the data, the metadata for the data including a Logical Block Address (LBA) and an offset for the data;
receiving the LBA and the offset from the KV-SSD; and
constructing the second command using the LBA and the offset for the data.
16. A method according to claim 13 , wherein associating the second command with a fabric command includes:
associating the second command with a second fabric command;
sending the second fabric command to a KV-SSD in a second system remote from the processor, the KV-SSD storing metadata for the data, the metadata for the data including an LBA and an offset for the data;
receiving the LBA and the offset from the KV-SSD in the second system remote from the processor; and
constructing the second command using the LBA and the offset for the data.
17. An article, comprising a non-transitory storage medium, the non-transitory storage medium having stored thereon instructions that, when executed by a machine, result in:
receiving a first command from a processor, the first command relating to a data;
generating a second command based on the first command;
inserting the second command in a memory-based fabric command; and
sending the fabric command to a storage device storing the data.
18. An article according to claim 17 , wherein sending the fabric command to a storage device storing the data includes sending the fabric command to a key-value (KV) Solid State Drive (SSD) in a system remote from the processor, the KV-SSD storing the data in an object.
19. An article according to claim 17 , wherein sending the fabric command to a storage device storing the data includes sending the fabric command to a Non-Volatile Memory Express (NVMe) SSD in a system remote from the processor, the NVMe SSD storing the data.
20. An article according to claim 19 , wherein sending the fabric command to a storage device storing the data further includes:
recovering the second command from the fabric command on the system remote from the processor;
constructing an NVMe command from the second command; and
executing the NVMe command on the NVMe SSD in the system remote from the processor.