Methods and apparatus for managing data flow in a continuous data replication system having journaling

1. A method, comprising:

receiving a data transfer request from a first component at a transmitter module in a continuous data replication system having a production site and a backup site, the production site comprises the transmitter module and a transmitter credit mechanism, the transmitter module is configured to transmit data over a network for replication in the backup site;

journaling replication data using a number of journal processing stages;

modifying a flow of credits from the transmitter credit mechanism to the first component to reduce or stop data flow to the transmitter module if a high-load is detected at the transmitter module;

modifying a flow of credits from a receiver credit mechanism to the transmitter module to reduce or stop data flow to the receiver module if a high-load condition is detected at a receiver module;

modifying a flow of credits from a distributor credit mechanism to the receiver module to reduce or stop data flow to the distributor module if a high-load condition is detected at a distributor module; and

reducing the number of journal processing stages for replicating data if a high-load condition is detected before activating the transmitter credit mechanism.

2. The method according to claim 1 , wherein the high-load condition at the transmitter module includes a transmit buffer filled to at least a selected amount of capacity.

3. The method according to claim 1 , wherein the distributor module controls journaling of the replication data.

4. The method according to claim 1 , further including activating the transmitter credit mechanism when a level of data transactions requested is greater than a selected amount.

5. The method according to claim 1 , wherein reducing further comprises reducing the number of journal processing stages in the journaling process by not including at least one undo processing stage.

6. The method according to claim 1 , further including activating the transmitter credit mechanism when a replication data stream is greater than a selected size.

7. The method according to claim 1 , wherein the transmitter credits from the transmitter credit mechanism define an amount of data that can be sent.

8. The method according to claim 1 , wherein the transmitter credits from the transmitter credit mechanism define an amount of data per unit time that can be sent.

9. The method according to claim 1 , further including performing backward data flow regulation where the distributor module monitors thread processing time for the journal processing stages of a multiple stage journaling process and reduces thread processing time for one or more of the journal processing stages.

10. The method according to claim 9 , further including reducing or stopping processing of at least a first one of the journal processing stages when a second one of the journal processing stages takes more than a selected amount of processing resources.

11. An article, comprising:

a storage medium having stored thereon instructions that when executed by a machine result in the following:

journaling replication data using a number of journal processing stages;

receiving a data transfer request from a first component at a transmitter module in a continuous data replication system having a production site and a backup site, the production site having the transmitter module and a transmitter credit mechanism, the transmitter module is configured to transmit data over a network for replication in the backup site;

modifying a flow of credits from the transmitter credit mechanism to the first component to reduce or stop data flow to the transmitter module if a high-load condition is detected at the receiver module;

modifying a flow of credits from a receiver credit mechanism to the transmitter module to reduce or stop data flow to the receiver module if a high-load condition is detected at a receiver module;

modifying a flow of credits from a distributor credit mechanism to the receiver module to reduce or stop data flow to the distributor module if a high-load condition is detected at a receiver module; and

reducing the number of journal processing stages for replicating data if a high-load condition is detected before activating the transmitter credit mechanism.

12. A data replication system, comprising:

a first data site having a transmitter module to receive data transactions;

a backup data site having a receiver module to receive the data transactions to be replicated and a distributor module to perform journaling of data from the receiver module, the system including a memory to store instructions to enable:

(a) receiving write data transactions from the first data site to be backed up at the backup data site;

(b) writing the write data to a first position in a do data stream;

(c) writing metadata for the write data to a first position in a do metadata stream, wherein the metadata for the write data in step (c) comprises a first field to store a pointer to an offset in the do data stream where the write data is located;

(d) reading data from a second position previously stored in the do data stream;

(e) reading metadata from a second position previously stored in the do metadata stream;

(f) reading data to be overwritten from a logical unit in the backup storage site, where location and size of the data to be overwritten is determined from the do metadata stream;

(g) writing the data to be overwritten to a first position in an undo data stream;

(h) writing metadata for the data to be overwritten to an undo metadata stream wherein the metadata for the data to be overwritten comprises a first field to store a pointer to an offset in the undo data stream where the data to be overwritten is located;

(i) writing the write data to the logical unit in the backup storage site; and

(j) managing a flow of credits between the transmitter module, the receiver module, and the distributor module to manage data flow in one or more of steps (a)-(i).

13. The system according to claim 12 , wherein the memory further stores additional instructions to determine whether a size of the do data stream is greater than a given value and to bypass, if the size of the do data stream is greater than the given value, the instructions to enable;

reading data to be overwritten from a logical unit in the backup storage site, where location and size of the data to be overwritten is determined from the do metadata stream;

writing the data to be overwritten to a first position in an undo data stream;

and writing metadata for the data to be overwritten to an undo metadata stream, the metadata for the data to be overwritten comprises a first field to store a pointer to an offset in the undo data stream where the data to be overwritten is located.