Asynchronous input and output for snapshots of virtual machines
A data management system comprises: a storage appliance configured to store a snapshot of a virtual machine; and one or more processors in communication with the storage appliance. The one or more processors are configured to perform operations including: identifying a plurality of shards of the virtual machine; requesting a snapshot of each of the plurality of shards; receiving the shards asynchronously; ordering the received snapshot shards sequentially into a results queue; and storing a single snapshot of the virtual machine based on the ordered snapshot shards. Operations may further include maintaining a flow control queue that limits the number of snapshot shards requested.
1. An apparatus, comprising:
a memory configured to store a snapshot of a virtual machine; and
one or more processors in communication with the memory, the one or more processors configured to cause the apparatus to:
determine a quantity of shards to be created for the virtual machine based at least in part on a network condition;
shard the virtual machine into a plurality of shards comprising the quantity of shards;
request shard snapshots corresponding to the plurality of shards;
receive, in response to the request, the shard snapshots to obtain received shard snapshots;
order the received shard snapshots to obtain ordered shard snapshots; and
store the snapshot of the virtual machine based on the ordered shard snapshots.
2. The apparatus of claim 1 , wherein, to request the shard snapshots, the one or more processors are configured to cause the apparatus to:
request a shard snapshot of each of the plurality of shards via a socket request using a virtual disk development kit (VDDK) library to a hypervisor via a single shared connection for each request.
3. The apparatus of claim 1 , wherein, to receive the shard snapshots, the one or more processors are configured to cause the apparatus to:
receive the shard snapshots asynchronously from a hypervisor through a virtual disk development kit (VDDK) library.
4. The apparatus of claim 3 , wherein, to order the received shard snapshots, the one or more processors are configured to cause the apparatus to:
order the received shard snapshots sequentially into a results queue.
5. The apparatus of claim 1 , wherein the one or more processors are further configured to cause the apparatus to:
maintain an offset-slot mapping indicating an ordering of the received shard snapshots.
6. The apparatus of claim 1 , wherein the one or more processors are further configured to cause the apparatus to:
maintain a flow control queue that limits a quantity of shard snapshots requested.
7. The apparatus of claim 6 , wherein the one or more processors are further configured to cause the apparatus to:
maintain a receive token queue; and
transfer tokens from the flow control queue to the receive token queue upon receiving respective shard snapshots of the plurality of shards.
8. The apparatus of claim 1 , wherein the one or more processors are further configured to cause the apparatus to:
store, based at least in part on receiving the shard snapshots, a shard snapshot that arrives before a token associated with the shard snapshot in an early receive map; and
update, after the token is received, an offset-slot mapping based at least in part on the shard snapshot in the early receive map.
9. The apparatus of claim 1 , wherein the one or more processors are further configured to cause the apparatus to:
present the ordered shard snapshots sequentially to a read application programming interface (API).
10. The apparatus of claim 1 , wherein, to receive the shard snapshots, the one or more processors are configured to cause the apparatus to:
receive the shard snapshots in accordance with a secure sockets layer (SSL) protocol.
11. The apparatus of claim 1 , wherein the one or more processors are configured to cause the apparatus to receive the shard snapshots asynchronously and order the shard snapshots sequentially.
12. A method, comprising:
determining a quantity of shards to be created for a virtual machine based at least in part on a network condition;
sharding, based at least in part on determining the quantity of shards, the virtual machine into a plurality of shards comprising the quantity of shards;
requesting, based at least in part on sharding the virtual machine, shard snapshots for the plurality of shards;
receiving, in response to requesting the shard snapshots, the shard snapshots for the plurality of shards;
ordering, based at least in part on receiving the shard snapshots, the shard snapshots; and
storing, based at least in part on ordering the shard snapshots, a snapshot of the virtual machine based on the shard snapshots.
13. The method of claim 12 , wherein requesting the shard snapshots comprises:
requesting a shard snapshot of each of the plurality of shards via a socket request using a virtual disk development kit (VDDK) library to a hypervisor via a single shared connection for each request.
14. The method of claim 12 , wherein receiving the shard snapshots comprises:
receiving the shard snapshots asynchronously from a hypervisor through a virtual disk development kit (VDDK) library.
15. The method of claim 14 , wherein ordering the shard snapshots comprises:
ordering the shard snapshots sequentially into a results queue.
16. The method of claim 12 , further comprising:
maintaining, based at least in part on receiving the shard snapshots, an offset-slot mapping indicating an ordering of the shard snapshots.
17. The method of claim 12 , further comprising:
maintaining a flow control queue that limits a quantity of shard snapshots requested.
18. The method of claim 17 , further comprising:
maintaining a receive token queue; and
transferring tokens from the flow control queue to the receive token queue upon receiving respective shard snapshots of the plurality of shards.
19. The method of claim 18 , further comprising:
storing, based at least in part on receiving the shard snapshots, a shard snapshot that arrives before a token associated with the shard snapshot in an early receive map; and
updating, after the token is received, an offset-slot mapping based at least in part on the shard snapshot in the early receive map.
20. A non-transitory, computer-readable medium storing code comprising instructions executable by a processor of an electronic device to cause the electronic device to:
determine a quantity of shards to be created for a virtual machine based at least in part on a network condition;
shard, based at least in part on determining the quantity of shards, the virtual machine into a plurality of shards comprising the quantity of shards;
request, based at least in part on sharding the virtual machine, shard snapshots for the plurality of shards;
receive, in response to requesting the shard snapshots, the shard snapshots for the plurality of shards;
order, based at least in part on receiving the shard snapshots, the shard snapshots; and
store, based at least in part on ordering the shard snapshots, a snapshot of the virtual machine based on the shard snapshots.