System and method for data set synchronization and replication

1. A method for synchronizing and replicating data sets, comprising:

receiving a request to synchronize a first data set associated with a first server and a second data set associated with a second server;

determining, with reference to one or more replication constraints, whether to begin synchronization, using a processor;

applying one or more resource control actions in response to determining to begin synchronization, wherein applying one or more resource control actions comprises generating one or more first low content replication journal entries comprising a set of instructions indicating write events at one or more data ranges;

retrieving data from the one or more data ranges;

generating one or more full content replication journal entries comprising the one or more first low content replication journal entries and the retrieved data;

determining whether to begin generating one or more online replication journal entries based on the amount of instructions in the one or more first low content replication journal entries, availability of a network, and the one or more replication constraints; and

generating the one or more online replication journal entries in response to a change in the first data set.

2. The method of claim 1 , wherein the one or more replication constraints are based on a system utilization metric of the first server.

3. The method of claim 1 , wherein the one or more replication constraints are based on the time of day.

4. The method of claim 1 , wherein applying one or more resource control actions comprises:

determining compliance with the one or more replication constraints; and

determining a synchronization speed based in response to non-compliance with the one or more replication constraints.

5. The method of claim 4 , wherein determining compliance with one or more replication constraints comprises:

determining a system utilization metric of the first server; and

comparing the system utilization metric of the first server to the one or more replication constraints.

6. The method of claim 1 , further comprising:

receiving a first set of instructions indicating write events at one or more data ranges from the first content journal entries;

receiving a second set of instructions indicating write events at one or more data ranges from the first low content journal entries;

identifying one or more redundancies between the first and second sets of instructions; and

generating a third set of instructions comprising the instructions of the first and second sets of instructions without the one or more redundancies.

7. The method of claim 1 , wherein it is determined to begin generating the one or more online replication journal entries, the method further comprising:

sending the one or more full content replication journal entries to the second server for replication, wherein the one or more online replication journal entries comprise a set of instructions indicating write events at one or more data ranges and the data to be written to the one or more data ranges; and

sending one or more of the one or more online replication journal entries to the second server for replication.

8. The method of claim 7 , further comprising:

detecting an error at the first server;

removing the data to be written to the one or more data ranges from the one or more online replication journal entries;

generating one or more second low content replication journal entries comprising a set of instructions indicating write events at one or more data ranges; and

consolidating the one or more online replication journal entries and the second set of one or more second low content replication journal entries.

9. The method of claim 8 , wherein consolidating the one or more online replication journal entries and the one or more second low content replication journal entries comprises:

receiving a first set of instructions indicating write events at one or more data ranges from the second low content journal entries;

receiving a second set of instructions indicating write events at one or more data ranges from the second low content journal entries;

identifying one or more redundancies between the first and second sets of instructions; and

generating a third set of instructions comprising the instructions of the first and second sets of instructions without the one or more redundancies.

10. The method of claim 1 , further comprising:

sending one or more of the one or more full content replication journal entries to the second server for replication; and

recording which of the one or more full content replication journal entries have been sent to the second server for replication.

11. A system for synchronizing and replicating data sets, comprising:

an interface operable to receive a request to synchronize a first data set associated with a first server and a second data set associated with a second server; and

one or more processors operable to:

determine, with reference to one or more replication constraints, whether to begin synchronization; and

apply one or more resource control actions in response to determining to begin synchronization, wherein applying one or more resource control actions comprises generating one or more first low content replication journal entries comprising a set of instructions indicating write events at one or more data ranges;

retrieve data from the one or more data ranges;

generate one or more full content replication journal entries comprising the one or more first low content replication journal entries and the retrieved data;

determine whether to begin generating one or more online replication journal entries based on the amount of instructions in the one or more first low content replication journal entries, availability of a network, and the one or more replication constraints; and

generate the one or more online replication journal entries in response to a change in the first data set.

12. The system of claim 11 , wherein the one or more processors operable to apply one or more resource control actions are further operable to:

determine compliance with one or more replication constraints, wherein at least one of the replication constraints is based on a system utilization metric or the time of day;

determine a synchronization speed based in response to non-compliance with the one or more replication constraints; and

generate one or more low content replication journal entries comprising a set of instructions indicating write events at one or more data ranges.

13. The system of claim 12 , wherein the one or more processors are further operable to:

receive a first set of instructions indicating write events at one or more data ranges from the low content journal entries;

receive a second set of instructions indicating write events at one or more data ranges from the low content journal entries;

identify one or more redundancies between the first and second sets of instructions; and

generate a third set of instructions comprising the instructions of the first and second sets of instructions without the one or more redundancies.

14. Logic encoded in a non-transitory computer readable medium, the logic being operable, when executed by a processor, to:

receive a request to synchronize a first data set associated with a first server and a second data set associated with a second server;

determine, with reference to one or more replication constraints, whether to begin synchronization;

apply one or more resource control actions in response to determining to begin synchronization, wherein applying one or more resource control actions comprises generating one or more first low content replication journal entries comprising a set of instructions indicating write events at one or more data ranges;

retrieve data from the one or more data ranges;

generate one or more full content replication journal entries comprising the one or more first low content replication journal entries and the retrieved data;

determine whether to begin generating one or more online replication journal entries based on the amount of instructions in the one or more first low content replication journal entries, availability of a network, and the one or more replication constraints; and

generate the one or more online replication journal entries in response to a change in the first data set.

15. The logic of claim 14 , wherein the logic operable to apply one or more resource control actions is further operable to:

determine compliance with one or more replication constraints, wherein at least one of the replication constraints is based on a system utilization metric or the time of day;

determine a synchronization speed based in response to non-compliance with the one or more replication constraints;

generate one or more low content replication journal entries comprising a set of instructions indicating write events at one or more data ranges;

retrieve data from the one or more data ranges; and

generate one or more full content replication journal entries comprising the one or more low content replication journal entries and the retrieved data.

16. The logic of claim 15 , the logic being further operable to:

receive a first set of instructions indicating write events at one or more data ranges from the low content journal entries;

receive a second set of instructions indicating write events at one or more data ranges from the low content journal entries;

identify one or more redundancies between the first and second sets of instructions; and

generate a third set of instructions comprising the instructions of the first and second sets of instructions without the one or more redundancies.

17. A system for synchronizing and replicating data sets, comprising:

means for receiving a request to synchronize a first data set associated with a first server and a second data set associated with a second server;

means for determining, with reference to one or more replication constraints, whether to begin synchronization;

means for applying one or more resource control actions in response to determining to begin synchronization, wherein applying one or more resource control actions comprises generating one or more first low content replication journal entries comprising a set of instructions indicating write events at one or more data ranges;

means for retrieving data from the one or more data ranges;

means for generating one or more full content replication journal entries comprising the one or more first low content replication journal entries and the retrieved data;

means for determining whether to begin generating one or more online replication journal entries based on the amount of instructions in the one or more first low content replication journal entries, availability of a network, and the one or more replication constraints; and

means for generating the one or more online replication journal entries in response to a change in the first data set.

18. The system of claim 17 , further comprising:

means for determining compliance with one or more replication constraints, wherein at least one of the replication constraints is based on a system utilization metric or the time of day;

means for determining a synchronization speed based in response to non-compliance with the one or more replication constraints;

means for generating one or more low content replication journal entries comprising a set of instructions indicating write events at one or more data ranges;

means for retrieving data from the one or more data ranges; and

means for generating one or more full content replication journal entries comprising the one or more low content replication journal entries and the retrieved data.

19. The system of claim 18 , further comprising:

means for receiving a first set of instructions indicating write events at one or more data ranges from the low content journal entries;

means for receiving a second set of instructions indicating write events at one or more data ranges from the low content journal entries;

means for identifying one or more redundancies between the first and second sets of instructions; and

means for generating a third set of instructions comprising the instructions of the first and second sets of instructions without the one or more redundancies.