Replication using deduplicated secondary copy data
An information management system according to certain aspects uses backup copies or other secondary copies of production data for the purposes of replicating production data to another client. The secondary copies can be deduplicated copies. By utilizing available secondary copies of the data for replication, the system can reduce the impact on the production machines associated with replication. Utilizing deduplicated copies not only reduces the amount of stored data, but also reduces the amount of data that is communicated between the source and the destination, increasing the speed of the replication process.
1. A system for replicating primary data from a source system to a destination system using secondary copy data in secondary storage, the system comprising:
one or more source secondary storage controller computers comprising hardware, the one or more source secondary storage controller computers configured to:
create a first deduplicated secondary copy of source primary data stored in a native format on the one or more source secondary storage devices, wherein the first deduplicated secondary copy is stored in a backup format that is different that the native format;
send signature values corresponding to a plurality of deduplicated data blocks in the first deduplicated secondary copy in one or more secondary data replication streams to one or more destination secondary storage controller computers, the signature values embedded in the one or more secondary replication streams contiguously with the plurality of deduplicated data blocks;
send command messages in the one or more secondary data replication streams to the one or more destination secondary storage controller computers, the command messages associated with the plurality of deduplicated data blocks, each command message having a timestamp associated therewith; and
one or more destination secondary storage controller computers comprising hardware, the one or more destination secondary storage controller computers configured to:
create a replicated deduplicated secondary copy based on the plurality of deduplicated data blocks from the first deduplicated secondary copy and the embedded signature values in the one or more secondary data replication streams, the replicated deduplicated secondary copy stored on the one or more destination secondary storage devices to create a second deduplicated secondary copy, the second deduplicated secondary copy stored in the backup format on the one or more destination secondary storage devices;
restore the second deduplicated secondary copy in the backup format to the one or more destination primary storage devices to create a restored version of the second deduplicated secondary copy, the restored version is in the native format of the source primary data; and
apply the command messages received in the one or more secondary data replication streams to the restored version of the second deduplicated secondary copy, based on the timestamps associated with the command messages to create replicated destination primary data consistent with the source primary data based on the timestamps, the replicated destination primary data stored in the one or more destination primary storage devices in the native format of the source primary data.
2. The system of claim 1 wherein the first deduplicated secondary copy is one of a plurality of deduplicated secondary copies that the one or more source secondary storage controllers are configured to perform as part of a continuous replication process.
3. The system of claim 2 wherein a first secondary data replication stream comprising the plurality of deduplicated data blocks from the first deduplicated secondary copy and the embedded signature values is sent separately from a second secondary data replication stream comprising the command messages.
4. The system of claim 2 wherein the plurality of deduplicated data blocks from the first deduplicated secondary copy, the embedded signature values and the commend messages are implemented as a single secondary data replication stream.
5. The system of claim 4 wherein each message command in the single secondary data replication stream comprises a header that indicates it is a command.
6. The system of claim 4 wherein a first data block in the single secondary data replication stream comprises a header that indicates it is a data block.
7. The system of claim 2 wherein the restore performed by the destination system is one of a plurality of restore operations, wherein each of the plurality of restore operations corresponds to one of the plurality of deduplicated secondary copies performed by the source system.
8. The system of claim 7 wherein the plurality of restore operations are performed at an interval such that changes to the source primary data are replicated to the destination system in a period of time not significantly greater than the interval.
9. The system of claim 1 wherein the plurality of deduplicated data blocks and embedded signature values in the one or more secondary data replication streams are stored in physically contiguous memory space.
10. The system of claim 1 wherein the plurality of deduplicated data blocks and embedded signature values in the one or more secondary data replication streams are stored in logically contiguous memory space.
11. A method of replicating primary data from a source system to a destination system using secondary copy data in secondary storage, the method comprising:
creating with one or more source secondary storage controllers a first deduplicated secondary copy that is a copy of source primary data on a source primary storage device, the source primary data stored in a native format and the first deduplicated secondary copy stored in a backup format that is different than the native format;
sending one or more secondary data replication streams with the one or more source secondary storage controllers, the one or more secondary data replication streams comprising a plurality of deduplicated data blocks from the first deduplicated secondary copy and embedded signature values associated therewith, the embedded signature values stored contiguously with the plurality of deduplicated data blocks associated therewith, wherein the one or more secondary data replication streams are sent to one or more destination secondary storage controller computers in the destination system; and
sending with the one or more source secondary storage controllers command messages in one or more secondary data replication streams to the one or more destination secondary storage controller computers, the command messages associated with the plurality of deduplicated data blocks, each command message having a timestamp associated therewith;
receiving with one or more destination secondary storage devices the one or more secondary data replication streams comprising the plurality of deduplicated data blocks and the embedded signature values, and creating at least one replicated deduplicated secondary copy, the replicated deduplicated secondary copy stored as a second deduplicated secondary copy in the backup format on the one or more destination secondary storage devices;
restoring the second deduplicated secondary copy in the backup format to one or more destination primary storage devices of the destination system to create a restored version of the second deduplicated secondary copy in the native format of the source primary data; and
applying the command messages received in the one or more secondary data replication streams to the restored version of the second deduplicated secondary copy, based on the timestamps associated with the command messages to create replicated destination primary data consistent with the source primary data based on the timestamps, the replicated destination primary data stored in the one or more destination primary storage devices in the native format of the source primary data.
12. The method of claim 11 wherein the first deduplicated secondary copy is one of a plurality of deduplicated secondary copies that the one or more source secondary storage controllers are configured to perform as part of a continuous replication process.
13. The method of claim 12 wherein a first secondary data replication stream comprising the plurality of deduplicated data blocks from the first deduplicated secondary copy and the embedded signature values is sent separately from a second secondary data replication stream comprising the command messages.
14. The method of claim 12 wherein the plurality of deduplicated data blocks from the first deduplicated secondary copy, the embedded signature values and the commend messages are implemented as a single secondary data replication stream.
15. The method of claim 14 wherein each message command in the single secondary data replication stream comprises a header that indicates it is a command.
16. The method of claim 14 wherein a first data block in the single secondary data replication stream comprises a header that indicates it is a data block.
17. The method of claim 12 wherein the restore performed by the destination system is one of a plurality of restore operations, wherein each of the plurality of restore operations corresponds to one of the plurality of deduplicated secondary copies performed by the source system.
18. The method of claim 17 wherein the plurality of restore operations are performed at an interval such that changes to the source primary data are replicated to the destination system in a period of time not significantly greater than the interval.
19. The method of claim 11 wherein the plurality of deduplicated data blocks and embedded signature values in the one or more secondary data replication streams are stored in physically contiguous memory space.
20. The method of claim 11 wherein the plurality of deduplicated data blocks and embedded signature values in the one or more secondary data replication streams are stored in logically contiguous memory space.