Using ephemeral storage as backing storage for journaling by a virtual storage system
Systems and methods for making use of non-persistent storage as the journaling storage media for a virtual storage system are provided. According to one embodiment, in order to meet the needs of Extreme Low Latency Workloads while also seeking to provide predictable performance and the lowest possible latency, ephemeral storage of the virtual storage system is used to preserve state information (e.g., in the form of boot arguments and an operation log journal) across a host failure recovery scenario in which the virtual storage system is expected to be redeployed within a compute instance brought up by a cloud environment of a hyperscaler on the same host, thereby providing improved data durability (fewer host failure scenarios that result in lost data) as compared to the use of ephemeral memory of the compute instance and lower write latency than the use of persistent storage provided by the cloud environment.
1. A virtual storage system comprising:
a processing resource; and
a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause the virtual storage system to:
during a boot process performed by the virtual storage system, discover ephemeral storage associated with a compute instance of a cloud environment in which the virtual storage system is deployed;
based on boot arguments stored on persistent storage associated with the compute instance and a unique identifier of the discovered ephemeral storage, determine a hosting status of the virtual storage system; and
responsive to the hosting status being indicative of the virtual storage system having been brought up on a same host of the cloud environment on which the virtual storage system was hosted prior to recovery from a failure, facilitate replay of an operation log journal that was maintained by the virtual storage system within the discovered ephemeral storage prior to the failure by registering the discovered ephemeral storage for use as a journaling storage media by the virtual storage system.
2. The virtual storage system of claim 1 , wherein the hosting status is indicative of the virtual storage system having been brought up on a same host, when the boot arguments include a previously stored unique identifier matching the unique identifier.
3. The virtual storage system of claim 1 , wherein the instructions further cause the virtual storage system to determine the virtual storage system has been rehosted by virtue of the boot arguments including a previously stored unique identifier of ephemeral storage that does not match the unique identifier.
4. The virtual storage system of claim 3 , wherein the instructions further cause the virtual storage system to facilitate maintenance of a new operation log journal by the virtual storage system within the discovered ephemeral storage by:
clearing the previously stored unique identifier from the boot arguments; and
causing the virtual storage system to persist the unique identifier within the boot arguments by rebooting the virtual storage system.
5. The virtual storage system of claim 1 , wherein the discovered ephemeral storage comprises a plurality of ephemeral disks that are directly attached to the same host.
6. The virtual storage system of claim 5 , wherein the instructions further cause the virtual storage system to:
select a first set of the plurality of ephemeral disks for use as a victim cache by the virtual storage system; and
select a second set of the plurality of ephemeral disks as backing storage for operation log journaling.
7. The virtual storage system of claim 6 , wherein the instructions further cause the virtual storage system to spread data blocks associated with the operation log journaling across the second set of the plurality of ephemeral disks by performing disk striping.
8. The virtual storage system of claim 1 , wherein the compute instance comprises a container.
9. The virtual storage system of claim 1 , wherein the compute instance comprises a virtual machine.
10. A method comprising:
during a boot process performed by a virtual storage system deployed within a compute instance of a cloud environment, discovering, by the virtual storage system, ephemeral storage associated with the compute instance;
based on boot arguments stored on persistent storage associated with the compute instance and a unique identifier of the discovered ephemeral storage, determining, by the virtual storage system, a hosting status of the virtual storage system; and
responsive to the hosting status of the virtual storage system being indicative of the virtual storage system having been brought up on a same host of the cloud environment on which the virtual storage system was hosted prior to recovery from a failure, facilitating replay of an operation log journal that was maintained by the virtual storage system within the discovered ephemeral storage prior to the failure by registering the discovered ephemeral storage for use as a journaling storage media by the virtual storage system.
11. The method of claim 10 , wherein the hosting status of the virtual storage system is indicative of the virtual storage system having been brought up on a same host, when the boot arguments include a previously stored unique identifier matching the unique identifier.
12. The method of claim 10 , further comprising determining the virtual storage system has been rehosted by virtue of the boot arguments including a previously stored unique identifier of ephemeral storage that does not match the unique identifier.
13. The method of claim 12 , further comprising facilitating maintenance of a new operation log journal by the virtual storage system within the discovered ephemeral storage by:
clearing the previously stored unique identifier from the boot arguments; and
causing the virtual storage system to persist the unique identifier within the boot arguments by rebooting the virtual storage system.
14. The method of claim 10 , wherein the discovered ephemeral storage comprises a plurality of ephemeral disks that are directly attached to the same host and wherein the method further comprises:
selecting a first set of the plurality of ephemeral disks for use as a victim cache by the virtual storage system;
selecting a second set of the plurality of ephemeral disks as backing storage for operation log journaling; and
spreading data blocks associated with the operation log journaling across the second set of the plurality of ephemeral disks by performing disk striping.
15. A non-transitory machine readable medium storing instructions, which when executed by a processing resource of a virtual storage system deployed within a compute instance of a cloud environment, cause the virtual storage system to:
during a boot process performed by the virtual storage system, discover ephemeral storage associated with the compute instance;
based on boot arguments stored on persistent storage associated with the compute instance and a unique identifier of the discovered ephemeral storage, determine a hosting status of the virtual storage system; and
responsive to the hosting status being indicative of the virtual storage system having been brought up on a same host of the cloud environment on which the virtual storage system was hosted prior to recovery from a failure, facilitate replay of an operation log journal that was maintained by the virtual storage system within the discovered ephemeral storage prior to the failure by registering the discovered ephemeral storage for use as a journaling storage media by the virtual storage system.
16. The non-transitory machine readable medium of claim 15 , wherein the hosting status is indicative of the virtual storage system having been brought up on a same host, when the boot arguments include a previously stored unique identifier matching the unique identifier.
17. The non-transitory machine readable medium of claim 15 , wherein the instructions further cause the virtual storage system to determine the virtual storage system has been rehosted by virtue of the boot arguments including a previously stored unique identifier of ephemeral storage that does not match the unique identifier.
18. The non-transitory machine readable medium of claim 17 , wherein the instructions further cause the virtual storage system to facilitate maintenance of a new operation log journal by the virtual storage system within the discovered ephemeral storage by:
clearing the previously stored unique identifier from the boot arguments; and
causing the virtual storage system to persist the unique identifier within the boot arguments by rebooting the virtual storage system.
19. The non-transitory machine readable medium of claim 15 , wherein the discovered ephemeral storage comprises a plurality of ephemeral disks that are directly attached to the same host and wherein the instructions further cause the virtual storage system to:
select a first set of the plurality of ephemeral disks for use as a victim cache by the virtual storage system;
select a second set of the plurality of ephemeral disks as backing storage for operation log journaling; and
spread data blocks associated with the operation log journaling across the second set of the plurality of ephemeral disks by performing disk striping.
20. The non-transitory machine readable medium of claim 15 , wherein the compute instance comprises a container or a comprises a virtual machine.