Any point in time backup on secondary storage for distributed consistency groups
Any point in time backups for distributed consistency is disclosed. IOs from a consistency group are received by multiple aggregators and stored in corresponding journals. In response to a bookmark, the multiple journals are synthesized to create a do stream or to add the multiple journals to the do stream. A full synchronization operation can be performed simultaneously with replication operations.
1. A method comprising:
in a distributed consistency group:
receiving a first IO stream at a primary aggregator from a splitter, wherein the splitter intercepts writes directed to a source;
receiving a second IO stream at a second aggregator from the splitter, wherein the splitter is configured to distribute the writes to the primary aggregator and the second aggregator;
receiving at the primary aggregator a bookmark in the first IO stream;
distributing the bookmark to the second aggregator;
writing a first journal of the primary aggregator to secondary storage and writing a second journal of the second aggregator to the secondary storage, wherein the first journal includes the first IO stream before the bookmark and the second journal includes the second IO stream before the bookmark; and
constructing a do stream by combining the first journal and the second journal and manipulating pointers such that entries in the do stream point to data in the first journal and to data in the second journal, wherein the do stream allows an any point in time backup to be generated for the consistency group.
2. The method of claim 1 , wherein the do stream includes an ordered list of IOs from the source.
3. The method of claim 2 , wherein the source is one of a virtual disk or a consistency group and wherein the writes are distributed in a round robin manner, based on address, based on volume, or in an alternating manner that is independent of address and wherein the splitter comprises multiple splitters.
4. The method of claim 1 , further comprising the primary aggregator notifying the second aggregator of the bookmark.
5. The method of claim 1 , wherein the bookmark identifies a point in time for the IO stream generated by the splitter.
6. The method of claim 1 , wherein the primary aggregator and the second aggregator transmit, respectively, the first journal and the second journal to the secondary storage, wherein the primary aggregator and the second aggregator each start a new journal for writes received in the first and second IO streams occurring after the bookmark.
7. The method of claim 1 , further comprising performing a full synchronization on the source, wherein the primary aggregator and the second aggregator are associated with different sequential portions of the source with regard to the full synchronization, wherein the primary aggregator and the second aggregator read from the source to generate corresponding sync journals for the full synchronization while receiving the first and second IO streams from the splitter.
8. The method of claim 7 , further comprising initiating the full synchronization by sending a full sync bookmark to the primary aggregator.
9. The method of claim 7 , further comprising synthesizing the sync journals and the first and second journals into the do stream.
10. The method of claim 2 , wherein the source is one of a virtual disk or a consistency group and wherein the writes are distributed in a round robin manner, based on address, based on volume, or in an alternating manner that is independent of address and wherein the splitter comprises multiple splitters.
11. A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:
in a distributed consistency group:
receiving a first IO stream at a primary aggregator from a splitter, wherein the splitter intercepts writes directed to a source;
receiving a second IO stream at a second aggregator from the splitter, wherein the splitter is configured to distribute the writes to the primary aggregator and the second aggregator;
receiving at the primary aggregator a bookmark in the first IO stream;
distributing the bookmark to the second aggregator;
writing a first journal of the primary aggregator to secondary storage and writing a second journal of the second aggregator to the secondary storage, wherein the first journal includes the first IO stream before the bookmark and the second journal includes the second IO stream before the bookmark; and
constructing a do stream by combining the first journal and the second journal and manipulating pointers such that entries in the do stream point to data in the first journal and to data in the second journal, wherein the do stream allows an any point in time backup to be generated for the consistency group.
12. The non-transitory storage medium of claim 11 , wherein the do stream includes an ordered list of IOs from the source.
13. The non-transitory storage medium of claim 11 , the operations further comprising the primary aggregator notifying the second aggregator of the bookmark.
14. The non-transitory storage medium of claim 11 , wherein the bookmark identifies a point in time for the IO stream generated by the splitter.
15. The non-transitory storage medium of claim 11 , wherein the primary aggregator and the second aggregator transmit, respectively, the first journal and the second journal to the secondary storage, wherein the primary aggregator and the second aggregator each start a new journal for writes received in the first and second IO streams occurring after the bookmark.
16. The non-transitory storage medium of claim 11 , the operations further comprising performing a full synchronization on the source, wherein the primary aggregator and the second aggregator are associated with a sequential portion of the source with regard to the full synchronization, wherein the primary aggregator and the second aggregator read from the source to generate corresponding sync journals for the full synchronization while receiving the first and second IO streams from the splitter.
17. The non-transitory storage medium of claim 16 , the operations further comprising initiating the full synchronization by sending a full sync bookmark to the primary aggregator.
18. The non-transitory storage medium of claim 16 , the operations further comprising synthesizing the corresponding journals and the first and second journals into the do stream.