Asynchronous data replication in a storage network
Methods and apparatus for asynchronous replication of data in a storage network. In an embodiment, a processing module(s) of a computing device identifies at least a first storage set and a second storage set for replicated storage of data. The processing module maintains a synchronization schedule for the first storage set and the second storage set. After initiating storage of a data object in the first storage set (e.g., using first error encoding parameters), the processing module determines, based at least in part on the synchronization schedule, to synchronize the first storage set and the second storage set. In response to determining to synchronize the first and second storage sets, the processing module determines that the second storage set requires the data object to maintain synchronization with the first storage set and facilitates storage of the data object in the second storage set (e.g., using second error encoding parameters).
1. A method for execution by one or more processing modules of a storage network, the method comprises:
identifying at least a first storage set and a second storage set for replicated storage of data;
maintaining a synchronization schedule for the first storage set and the second storage set;
initiating storage of a data object in the first storage set, including:
encoding the data object with first dispersed storage error encoding parameters to generate a first plurality of encoded data slices; and
sending the first plurality of encoded data slices to the first storage set;
determining, based at least in part on the synchronization schedule, to synchronize the first storage set and the second storage set;
determining that the second storage set requires the data object to maintain synchronization with the first storage set; and
in response to determining that the second storage set requires the data object, facilitating storage of the data object in the second storage set, including:
retrieving the first plurality of encoded data slices from the first storage set;
decoding the first plurality of encoded data slices, using the first dispersed storage error encoding parameters, to reproduce the data object;
encoding the reproduced data object, using second dispersed storage error encoding parameters, to generate a second plurality of encoded data slices; and
sending the second plurality of encoded data slices to the second storage set for storage therein.
2. The method of claim 1 , further comprising:
maintaining a synchronization log that tracks the status of write operations to the first storage set and the second storage set,
wherein determining that the second storage set requires the data object to maintain synchronization with the first storage set is based, at least in part, on the synchronization log.
3. The method of claim 2 , wherein maintaining the synchronization log includes updating the synchronization log in response to detecting a failure to store a portion of the data object in the second storage set.
4. The method of claim 1 , wherein facilitating storage of the data object in the second storage set includes:
retrieving the data object from the first storage set; and
sending the data object to the second storage set for storage therein.
5. The method of claim 1 , wherein determining to synchronize the first storage set and the second storage set is further based on detecting a return to availability of the second storage set after a period of unavailability.
6. The method of claim 1 , wherein determining to synchronize the first storage set and the second storage set is further based on detecting a return to availability of the first storage set after a period of unavailability.
7. The method of claim 1 , wherein the first dispersed storage error encoding parameters differ from the second dispersed storage error encoding parameters.
8. The method of claim 1 , wherein facilitating storage of the data object in the second storage set includes performing an integrity check on the data object.
9. A computing device comprises:
at least one interface;
memory that stores operational instructions; and
one or more processing modules operably coupled to the at least one interface and the memory, wherein the one or more processing modules are configured to execute the operational instructions to:
identify at least a first storage set and a second storage set for replicated storage of data;
maintain a synchronization schedule for the first storage set and the second storage set;
initiate storage of a data object in the first storage set, including:
encoding the data object with first dispersed storage error encoding parameters to generate a first plurality of encoded data slices; and
sending, via the at least one interface, the first plurality of encoded data slices to the first storage set;
determine, based at least in part on the synchronization schedule, to synchronize the first storage set and the second storage set;
determine that the second storage set requires the data object to maintain synchronization with the first storage set; and
in response to determining that the second storage set requires the data object, facilitate storage of the data object in the second storage set, including:
retrieving, via the at least one interface, the first plurality of encoded data slices from the first storage set;
decoding the first plurality of encoded data slices, using the first dispersed storage error encoding parameters, to reproduce the data object;
encoding the reproduced data object, using second dispersed storage error encoding parameters, to generate a second plurality of encoded data slices; and
sending, via the at least one interface, the second plurality of encoded data slices to the second storage set for storage therein.
10. The computing device of claim 9 , wherein the one or more processing modules are further configured to execute the operational instructions to:
maintain a synchronization log that tracks the status of write operations to the first storage set and the second storage set,
wherein determining that the second storage set requires the data object to maintain synchronization with the first storage set is based, at least in part, on the synchronization log.
11. The computing device of claim 10 , wherein maintaining the synchronization log includes updating the synchronization log in response to detecting a failure to store a portion of the data object in the second storage set.
12. The computing device of claim 10 , wherein facilitating storage of the data object in the second storage set includes performing an integrity check on the data object.
13. The computing device of claim 9 , wherein facilitating storage of the data object in the second storage set includes:
retrieving, via the at least one interface, the data object from the first storage set; and
sending, via the at least one interface, the data object to the second storage set for storage therein.
14. The computing device of claim 9 , wherein determining to synchronize the first storage set and the second storage set is further based on detecting a return to availability of the second storage set after a period of unavailability.
15. The computing device of claim 9 , wherein determining to synchronize the first storage set and the second storage set is further based on detecting a return to availability of the first storage set after a period of unavailability.
16. The computing device of claim 9 , wherein the first dispersed storage error encoding parameters differ from the second dispersed storage error encoding parameters.
17. A computer readable storage medium comprises:
at least one non-transitory memory section that stores operational instructions that, when executed by one or more processing modules of a computing device of a storage network, causes the computing device to:
identify at least a first storage set and a second storage set for replicated storage of data;
maintain a synchronization schedule for the first storage set and the second storage set;
initiate storage of a data object in the first storage set, including:
encoding the data object with first dispersed storage error encoding parameters to generate a first plurality of encoded data slices; and
sending the first plurality of encoded data slices to the first storage set;
determine, based at least in part on the synchronization schedule, to synchronize the first storage set and the second storage set;
determine that the second storage set requires the data object to maintain synchronization with the first storage set; and
in response to determining that the second storage set requires the data object, facilitate storage of the data object in the second storage set, including:
retrieving the first plurality of encoded data slices from the first storage set;
decoding the first plurality of encoded data slices, using the first dispersed storage error encoding parameters, to reproduce the data object;
encoding the reproduced data object, using second dispersed storage error encoding parameters, to generate a second plurality of encoded data slices; and
sending the second plurality of encoded data slices to the second storage set for storage therein.
18. The computer readable storage medium of claim 17 , wherein the at least one non-transitory memory section stores additional operational instructions that, when executed by the one or more processing modules of the computing device, causes the computing device to:
maintain a synchronization log that tracks the status of write operations to the first storage set and the second storage set,
wherein determining that the second storage set requires the data object to maintain synchronization with the first storage set is based, at least in part, on the synchronization log.
19. The computer readable storage medium of claim 18 , wherein maintaining the synchronization log includes updating the synchronization log in response to detecting a failure to store a portion of the data object in the second storage set.