Distributed storage array
View Patent ↗Systems and methods for asynchronous backup of virtual disks in a distributed storage array are disclosed. An exemplary method may comprise receiving an IO stream at the virtual disks in the distributed storage array, writing data to one or more storage cells of the virtual disks in the distributed storage array, and logging completed writes in a first batch for each of the one or more storage cells. The method also comprises establishing a consistency point in the IO stream, terminating logging to the first batch for each of the one or more storage cells at the consistency point and logging completed writes after the consistency point to a new batch for each of the one or more storage cells. The method also comprises transferring writes logged in the first batch to at least one remote storage so that the at least one remote storage is crash consistent.
1. A method for batch transition during asynchronous backup of a distributed storage array, comprising:
querying a plurality of storage cells in the distributed storage array for a current Batch ID;
broadcasting the largest Batch ID incremented by one (M+1) to the plurality of storage cells in the distributed storage array;
suspending cache commits for pending and incoming writes at the plurality of storage cells to establish a consistency point; and
setting the Batch ID at the plurality of storage cells to the largest Batch ID incremented by one (M+1) for recording the pending and incoming writes in a new batch after the consistency point.
2. The method of claim further 1 , comprising resuming cache commits to the new batch after establishing the consistency point.
3. The method of claim 1 , further comprising establishing the consistency point only after a quorum of storage cells acknowledges that cache commits for the pending and incoming writes are suspended.
4. The method of claim 3 , resuming cache commits for the pending and incoming writes without incrementing the Batch ID if the predetermined number of storage cells fails to acknowledge that the pending and incoming writes are suspended.
5. The method of claim 3 , further comprising timing out the suspending of cache commits if a batch manager fails.
6. A method for batch transition during asynchronous backup of virtual disks in a distributed storage array, comprising:
querying at least a quorum of storage cells in the distributed storage array for a current Batch ID;
broadcasting the largest Batch ID incremented by one (M+1) along with a transition time to at least the quorum of storage cells in the distributed storage array;
setting the Batch ID for at least the quorum of storage cells to the largest Batch ID incremented by one (M+1) at the transition time;
logging completed writes having a timestamp prior to the transition time in an old batch; and
logging completed writes having a timestamp at or after the transition time in a new batch.
7. The method of claim 6 , further comprising suspending write acknowledgements to one or more clients for a time equal to the maximum skew between any two clocks in the distributed storage array for the transition time.
8. The method of claim 6 , further comprising resuming write acknowledgements to one or more clients after the transition time.
9. The method of claim 6 , further comprising acknowledging a successful batch transition after the transition time.
10. The method of claim 6 , wherein logging completed writes in the new batch is only after a predetermined number of storage cells acknowledge receipt of the transition time.
11. The method of claim 10 , further comprising resuming logging to the old batch if the predetermined number of storage cells fails to acknowledge receipt of the transition time.