Facilitating multiple device consumption of shared namespaces of ephemeral storage devices by a consumer of a virtual storage device
Systems and methods for multiple device consumption of shared namespaces of ephemeral storage devices by a consumer of a virtual storage system are provided. In an example, multiple namespaces of respective ephemeral storage devices are shared among multiple consumers of a virtual storage system by creating multiple partitions within each of the namespaces for use by respective consumers of the multiple consumers. Corresponding partitions of respective shared namespace may then be treated as a stripe set to facilitate multiple device consumption for a subsystem (e.g., operation log journaling) of the virtual storage system by striping data associated with input/output (I/O) requests of a consumer (e.g., a journaling driver) across one or more stripe units of one or more stripes within the stripe set.
1 . A method comprising:
exposing, by a storage driver of a virtual storage system deployed within a compute instance of a cloud environment, an interface through which one or more consumers of the virtual storage system access a plurality of ephemeral storage devices associated with the compute instance; and
facilitating increased write performance achieved by the one or more consumers by:
based on one or more reservation requests received via the interface from a first consumer of the one or more consumers, creating, by the storage driver, a first stripe set for use by the first consumer including a first partition within a first namespace of a first ephemeral storage device of the plurality of ephemeral storage devices and a second partition within a second namespace of a second ephemeral storage device of the plurality of ephemeral storage devices; and
striping data associated with a write request by the first consumer across one or more stripe units of one or more stripes within the first partition and the second partition.
2 . The method of claim 1 , further comprising:
based on one or more reservation requests received via the interface from a second consumer of the one or more consumers, creating, by the storage driver, a second stripe set for use by the second consumer including a third partition within the first namespace and a fourth partition within the second namespace; and
striping data associated with a write request by the second consumer across one or more stripe units of one or more stripes within the third partition and the fourth partition.
3 . The method of claim 2 , wherein the first consumer comprises a journaling driver, wherein the first partition and the second partition are used as a backing store for an operation log journal, wherein the second consumer comprises a victim cache driver, and wherein the third partition and the fourth partition are used as a backing store for a victim cache.
4 . The method of claim 3 , wherein the storage driver comprises a non-volatile memory express (NVMe) driver and the interface comprises and application programming interface.
5 . The method of claim 3 , further comprising prioritizing input/output (I/O) requests received by the storage driver from the journaling driver over I/O requests received by the storage driver from the victim cache driver.
6 . The method of claim 5 , wherein said prioritizing includes giving priority to write operations performed by the journaling driver over read operations performed by the victim cache driver.
7 . The method of claim 5 , wherein said prioritizing includes imposing one or both of an outstanding I/O operation limit and a throughput limit on one or both of read operations and write operations performed by the victim cache driver.
8 . The method of claim 3 , wherein the plurality of ephemeral storage devices comprise direct-attached-storage that are internal to a host on which the compute instance is deployed.
9 . The method of claim 3 , wherein the plurality of ephemeral storage devices are external to a host on which the compute instance is deployed.
10 . A storage system comprising:
one or more processing resources; and
instructions that when executed by the one or more processing resources cause the storage system to:
share a plurality of namespaces of respective ephemeral storage devices among a plurality of consumers of a storage system by creating a plurality of partitions within each of the plurality of namespaces for use by respective consumers of the plurality of consumers, wherein a first partition of the plurality of partitions within a first namespace of the plurality of namespaces and a second partition of the plurality of partitions within a second namespace of the plurality of namespaces are part of a first stripe set; and
avoid processing latency associated with use of an operating system (OS)-provided stripe mechanism by bypassing the OS-provided stripe mechanism during performance of an input/output (I/O) operation on the first stripe set based on a request by a first consumer of the plurality of consumers.
11 . The storage system of claim 10 , wherein the instructions further cause the storage system to configure, initialize, and manage the respective ephemeral storage devices via the OS-provided stripe mechanism.
12 . The storage system of claim 10 , wherein said bypassing includes a non-volatile memory express (NVMe) driver of the storage system fitting sub-requests within one or more slice units of the first stripe set by dividing the I/O operation into the sub-requests.
13 . The storage system of claim 10 , wherein the first consumer comprises a journaling driver and wherein the first partition and the second partition are used as a backing store for an operation log journal.
14 . The storage system of claim 10 , wherein the first partition and the second partition are created via an application programming interface (API) exposed by a non-volatile memory express (NVMe) driver of the storage system.
15 . The storage system of claim 10 , wherein a third partition and a fourth partition are part of a second stripe set, and wherein the instructions further cause the storage system to stripe data associated with a write request by the second consumer across one or more stripe units of one or more stripes within the third partition and the fourth partition.
16 . The storage system of claim 15 , wherein the second consumer comprises a victim cache driver and wherein the third partition and the fourth partition are used as a backing store for a victim cache.
17 . A non-transitory machine readable medium storing instructions, which when executed by one or more processing resources of a virtual storage system deployed within a compute instance of a cloud environment, cause the virtual storage system to:
expose, by a non-volatile memory express (NVMe) driver of the virtual storage system, an application programming interface (API) through which one or more consumers of the virtual storage system access a plurality of ephemeral storage devices associated with the compute instance;
based on one or more reservation requests received via the API from a first consumer of the one or more consumers, create, by the NVMe driver, a first stripe set for use by the first consumer including a first partition within a first namespace of a first ephemeral storage device of the plurality of ephemeral storage devices and a second partition within a second namespace of a second ephemeral storage device of the plurality of ephemeral storage devices; and
avoid processing latency associated with use of an operating system (OS)-provided stripe mechanism by bypassing the OS-provided stripe mechanism during performance of an input/output (I/O) operation on the first stripe set based on a request by a first consumer of the plurality of consumers.
18 . The non-transitory machine readable medium of claim 17 , wherein the performance of the I/O operation includes striping data associated with a write request by the first consumer across one or more stripe units of one or more stripes within the first partition and the second partition.
19 . The non-transitory machine readable medium of claim 17 , wherein the first consumer comprises a journaling driver, wherein the first partition and the second partition are used as a backing store for an operation log journal, wherein the second consumer comprises a victim cache driver, wherein a third partition and a fourth partition are part of a second stripe set, and wherein the third partition and the fourth partition are used as a backing store for a victim cache.
20 . The non-transitory machine readable medium of claim 19 , wherein the instructions further cause the virtual storage system to prioritizing input/output (I/O) requests received by the NVMe driver from the journaling driver over I/O requests received by the NVMe driver from the victim cache driver.