Methods and apparatus for optimal journaling for continuous data replication
View Patent ↗Methods and systems for continuous data replication, including receiving a write transaction from a production site, the write transaction including raw data (RWNEW), a size indicator (SZ 1 ) for the size of the raw data, and a start address (ADD 1 ) for writing the raw data, copying the raw data (RWNEW) into a first journal data stream at a current journal address (JADD 1 ), and copying the size indicator (SZ 1 ), the start address (ADD 1 ) and the journal address (JADD 1 ) into a first journal meta-data stream.
1. A method for continuous data replication, comprising:
receiving a write transaction from a production site, the write transaction including first raw data (RWNEW), a first size indicator (SZ 1 ) for the size of the raw data, and a first start address (ADD 1 ) for writing the first raw data;
copying the first raw data (RWNEW) into a first journal data stream starting at a first current journal address (JADD 1 ); and
copying the first size indicator (SZ 1 ), the first start address (ADD 1 ), and the first current journal address (JADD 1 ) into a first journal meta-data stream.
2. The method according to claim 1 , further including copying the first size indicator (SZ 1 ), the first start address (ADD 1 ), and the first current journal address (JADD 1 ) into the end of the first journal meta-data stream.
3. The method of claim 2 further comprising:
reading a second size indicator (SZ 2 ), a second start address (ADD 2 ), and a second journal address (JADD), from the beginning of the first journal meta-data stream;
reading second raw data (RW) from the first journal data stream beginning at the second journal address (JADD) and extending to a size of the second size indicator (SZ 2 ); and
copying the second raw data (RW) to a storage unit, beginning at a third start address (ADD).
4. The method of claim 3 , further comprising, prior to the copying of the second raw data (RW) to the storage unit:
reading third raw data (RWOLD) from the storage unit beginning at the second start address (ADD 2 ) and extending to a size of the second size indicator (SZ 2 );
copying the third raw data (RWOLD) to an undo journal data stream at the second current journal address (JADD 2 ); and
copying size the second indicator (SZ 2 ), the second start address (ADD 2 ), and the third journal address (JADD 2 ) into an undo journal meta-data stream, at the end of the undo journal meta-data stream.
5. The method of claim 3 , further comprising, prior to the copying of the second raw data (RW) to the storage unit:
conditionally reading the third raw data (RWOLD), from the storage unit beginning at the second start address (ADD 2 ) and extending to a size of the second size indicator (SZ 2 ) if a size of the first journal data stream exceeds a selected length;
conditionally copying the third raw data (RWOLD) to a second journal data stream at the third current journal address (JADD 2 ) if the size of the first journal data stream exceeds the selected length; and
conditionally copying the second size indicator (SZ 2 ), the second start address (ADD 2 ), and the third journal address (JADD 2 ) into a second journal meta-data stream, at the end of the second journal meta-data stream, if the size of the first journal data stream exceeds the selected length.
6. The method of claim 5 wherein the selected length corresponds to a percentage of maximum capacity for the first journal data stream.
7. The method of claim 5 wherein the selected length corresponds to a recovery time.
8. An article comprising:
a computer-readable storage medium having stored thereon instructions that when executed by a machine result in the following:
receiving a write transaction from a production site, the write transaction including first raw data (RWNEW), a first size indicator (SZ 1 ) for the size of the raw data, and a first start address (ADD 1 ) for writing the first raw data;
copying the first raw data (RWNEW) into a first journal data stream starting at a first current journal address (JADD 1 ); and
copying the first size indicator (SZ 1 ), the first start address (ADD 1 ), and the first current journal address (JADD 1 ) into a first journal meta-data stream.
9. The article according to claim 8 , further including instructions to enable copying the first size indicator (SZ 1 ), the first start address (ADD 1 ), and the first current journal address (JADD 1 ) into the end of the first journal meta-data stream.
10. The article of claim 9 further including instructions to enable:
reading a second size indicator (SZ 2 ), a second start address (ADD 2 ), and a second journal address (JADD), from the beginning of the first journal meta-data stream;
reading second raw data (RW) from the first journal data stream beginning at the second journal address (JADD) and extending to a size of the second size indicator (SZ 2 ); and
copying the second raw data (RW) to a storage unit, beginning at a third start address (ADD).
11. The article of claim 10 , further including instructions to enable, prior to the copying of the second raw data (RW) to the storage unit:
conditionally reading the third raw data (RWOLD), from the storage unit beginning at the second start address (ADD 2 ) and extending to a size of the second size indicator (SZ 2 ) if a size of the first journal data stream exceeds a selected length;
conditionally copying the third raw data (RWOLD) to a second journal data stream at the third current journal address (JADD 2 ) if the size of the first journal data stream exceeds the selected length; and
conditionally copying the second size indicator (SZ 2 ), the second start address (ADD 2 ), and the third journal address (JADD 2 ) into a second journal meta-data stream, at the end of the second journal meta-data stream, if the size of the first journal data stream exceeds the selected length.
12. The article of claim 11 , wherein the selected length corresponds to a percentage of maximum capacity for the first journal data stream.
13. The article of claim 11 wherein the selected length corresponds to a recovery time.
14. The article of claim 9 , further including instructions to enable, prior to the copying of the second raw data (RW) to the storage unit:
reading third raw data (RWOLD) from the storage unit beginning at the second start address (ADD 2 ) and extending to a size of the second size indicator (SZ 2 );
copying the third raw data (RWOLD) to an undo journal data stream at the second current journal address (JADD 2 ); and
copying size the second indicator (SZ 2 ), the second start address (ADD 2 ), and the third journal address (JADD 2 ) into an undo journal meta-data stream, at the end of the undo journal meta-data stream.
15. A system, comprising:
a source side storage system to store write transaction data, the source side storage system including a source data protection appliance coupled to a source journal logical unit and a source side logical unit;
a target side storage system to replicate write transaction data stored in the source side storage system, the target side storage system including a target data protection appliance coupled to a target journal logical unit;
the target side storage system having a memory for storing instructions to enable:
receiving a write transaction including first raw data (RWNEW), a first size indicator (SZ 1 ) for the size of the raw data, and a first start address (ADD 1 ) for writing the first raw data;
copying the first raw data (RWNEW) into a first journal data stream starting at a first current journal address (JADD 1 ); and
copying the first size indicator (SZ 1 ), the first start address (ADD 1 ), and the first current journal address (JADD 1 ) into a first journal meta-data stream.
16. The system according to claim 15 , further including instructions to enable copying the first size indicator (SZ 1 ), the first start address (ADD 1 ), and the first current journal address (JADD 1 ) into the end of the first journal meta-data stream.
17. The system of claim 16 further including instructions to enable:
reading a second size indicator (SZ 2 ), a second start address (ADD 2 ), and a second journal address (JADD), from the beginning of the first journal meta-data stream;
reading second raw data (RW) from the first journal data stream beginning at the second journal address (JADD) and extending to a size of the second size indicator (SZ 2 ); and
copying the second raw data (RW) to a storage unit, beginning at a third start address (ADD).
18. The system of claim 17 , further including instructions to enable, prior to the copying of the second raw data (RW) to the storage unit:
reading third raw data (RWOLD) from the storage unit beginning at the second start address (ADD 2 ) and extending to a size of the second size indicator (SZ 2 );
copying the third raw data (RWOLD) to an undo journal data stream at the second current journal address (JADD 2 ); and
copying size the second indicator (SZ 2 ), the second start address (ADD 2 ), and the third journal address (JADD 2 ) into an undo journal meta-data stream, at the end of the undo journal meta-data stream.
19. The system of claim 18 , further including instructions to enable, prior to the copying of the second raw data (RW) to the storage unit:
conditionally reading the third raw data (RWOLD), from the storage unit beginning at the second start address (ADD 2 ) and extending to a size of the second size indicator (SZ 2 ) if a size of the first journal data stream exceeds a selected length;
conditionally copying the third raw data (RWOLD) to a second journal data stream at the third current journal address (JADD 2 ) if the size of the first journal data stream exceeds the selected length; and
conditionally copying the second size indicator (SZ 2 ), the second start address (ADD 2 ), and the third journal address (JADD 2 ) into a second journal meta-data stream, at the end of the second journal meta-data stream, if the size of the first journal data stream exceeds the selected length.
20. The system of claim 19 wherein the selected length corresponds to a percentage of maximum capacity for the first journal data stream and/or a recovery time.