Using ephemeral storage as backing storage for journaling by a virtual storage system
Systems and methods for making use of non-persistent storage as the journaling storage media for a virtual storage system are provided. According to one embodiment, in order to meet the needs of Extreme Low Latency Workloads while also seeking to provide predictable performance and the lowest possible latency, ephemeral storage of the virtual storage system is used to preserve state information (e.g., in the form of boot arguments and an operation log journal) across a host failure recovery scenario in which the virtual storage system is expected to be redeployed within a compute instance brought up by a cloud environment of a hyperscaler on the same host, thereby providing improved data durability (fewer host failure scenarios that result in lost data) as compared to the use of ephemeral memory of the compute instance and lower write latency than the use of persistent storage provided by the cloud environment.
1. A method comprising:
discovering, by a virtual storage system deployed within a compute instance of a host of a public cloud, one or more direct-attached storage (DAS) devices of the host that are associated with the compute instance;
determining, by the virtual storage system, the virtual storage system has been brought up on the host after recovery from a failure; and
regaining access to an operation log that was maintained by the virtual storage system within the discovered one or more DAS devices prior to the failure by registering at least a subset of the discovered one or more DAS devices for use as a journaling storage media by the virtual storage system.
2. The method of claim 1 , wherein said discovering is performed during a boot process of the virtual storage system.
3. The method of claim 1 , wherein said determining is based on comparing one or more arguments retrieved from persistent storage associated with the compute instance and one or more unique identifiers of the discovered one or more DAS devices.
4. The method of claim 1 , wherein the subset of the discovered one or more DAS devices includes at least two solid-state drives (SSDs) and wherein the method further comprises spreading data blocks associated with the operation log across the at least two SSDs by performing disk striping.
5. The method of claim 1 , wherein the compute instance comprises a virtual machine.
6. The method of claim 1 , wherein the compute instance comprises a container.
7. A virtual storage system comprising:
a processing resource; and
a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause the virtual storage system to:
discover one or more direct-attached storage (DAS) devices of a host that are associated with a compute instance of a public cloud in which the virtual storage system is deployed;
determine a hosting status of the virtual storage system; and
when the hosting status is indicative of the virtual storage system having been brought up on the host on which the virtual storage system was hosted prior to recovery from a failure, regain access to an operation log that was maintained by the virtual storage system within the one or more DAS devices prior to the failure by registering at least a subset of the one or more DAS devices for use as a journaling storage media by the virtual storage system.
8. The virtual storage system of claim 7 , wherein the one or more DAS devices comprise solid-state drives (SDDs).
9. The virtual storage system of claim 7 , wherein the instructions further cause the virtual storage system to facilitate maintenance of a new operation log by the virtual storage system within the subset of the one or more DAS devices when the hosting status is indicative of the virtual storage system having been rehosted.
10. The virtual storage system of claim 7 , wherein discovery of the one or more DAS devices is performed during a boot process of the virtual storage system.
11. The virtual storage system of claim 7 , wherein the hosting status is determined based on comparing one or more arguments retrieved from persistent storage associated with the compute instance and a unique identifier of a DAS device of the one or more DAS devices.
12. The virtual storage system of claim 7 , wherein the instructions further cause virtual storage system to spread data blocks associated with the operation log across the subset of the one or more DAS devices by performing disk striping.
13. The virtual storage system of claim 7 , wherein the compute instance comprises a virtual machine.
14. The virtual storage system of claim 7 , wherein the compute instance comprises a container.
15. A non-transitory machine readable medium storing instructions, which when executed by a processing resource of a virtual storage system deployed within a compute instance of a host of a public cloud, cause the virtual storage system to:
discover one or more direct-attached storage (DAS) devices of the host that are associated with the compute instance;
determine a hosting status of the virtual storage system; and
when the hosting status is indicative of the virtual storage system having been brought up on the host on which the virtual storage system was hosted prior to recovery from a failure, regain access to an operation log that was maintained by the virtual storage system within the one or more DAS devices prior to the failure by registering at least a subset of the one or more DAS devices for use as a journaling storage media by the virtual storage system.
16. The non-transitory machine readable medium of claim 15 , wherein the instructions further cause the virtual storage system to facilitate maintenance of a new operation log by the virtual storage system within the subset of the one or more DAS devices when the hosting status is indicative of the virtual storage system having been rehosted.
17. The non-transitory machine readable medium of claim 15 , wherein discovery of the one or more DAS devices is performed during a boot process of the virtual storage system.
18. The non-transitory machine readable medium of claim 15 , wherein the hosting status is determined based on comparing one or more arguments retrieved from persistent storage associated with the compute instance and a unique identifier of a DAS device of the one or more DAS devices.
19. The non-transitory machine readable medium of claim 15 , wherein the subset of the one or more DAS devices includes at least two solid-state drives (SSDs) and wherein the instructions further cause the virtual storage system to spread data blocks associated with the operation log across the at least two SSDs by performing disk striping.
20. The non-transitory machine readable medium of claim 15 , wherein the compute instance comprises a virtual machine or a container.