Token-based offload data transfer with synchronous replication
A method is provided, comprising: receiving, at a source system, a first copy instruction, the first copy instruction being associated with a token that represents one or more data items, the first copy instruction instructing the source system to copy the one or more data items from a first volume to a second volume; in response to the first copy instruction, retrieving one or more hash digests from a snapshot that is associated with the token, each of the one or more hash digests being associated with a different one of the one or more data items; and transmitting, to a target system, a second copy instruction that is associated with the one or more hash digests, the second copy instruction instructing the target system to copy the one or more data items to a replica of the second volume that is stored at the target system.
1. A method comprising:
receiving, at a source system, a first copy instruction, the first copy instruction including or referencing a token that represents one or more data items, the token containing an identifier of a token-specific snapshot that is arranged to preserve the one or more data items, the first copy instruction instructing the source system to copy the one or more data items from a first volume to a second volume;
detecting whether the source system is currently performing synchronous replication of the first volume and the second volume;
when the source system is performing synchronous replication of both the first volume and the second volume: (i) identifying the token-specific snapshot, (ii) retrieving one or more hash digests from the token-specific snapshot, each of the one or more hash digests being associated with a different one of the one or more data items, (iii) transmitting, to a target system, a second copy instruction that is associated with the one or more hash digests, the second copy instruction instructing the target system to copy the one or more data items to a replica of the second volume that is stored at the target system, and (iv) copying the one or more data items from the first volume to the second volume; and
when the source system is not performing synchronous replication of the second volume, copying the one or more data items from the first volume to the second volume,
wherein the token-specific snapshot is created in response to a request to create the token for the purpose of preserving the one or more data items that are represented by the token, the token-specific snapshot being created before the first copy instruction is received,
wherein the source system and the target system are part of a storage system, and the token-specific snapshot is subject to one or more policies in the storage system that limit access to the token-specific snapshot to accesses that are performed for a purpose of performing token-based operations, and
wherein the storage system is configured to create a snapshot of the first volume that corresponds to unmapped read or write commands, and which is subject to different access restrictions than the token-specific snapshot, the snapshot that corresponds to unmapped read or write commands being different from the token-specific snapshot.
2. The method of claim 1 , further comprising, when the source system is performing synchronous replication of the second volume, but not of the first volume, returning an error.
3. The method of claim 1 , wherein the source system and the target system are part of a content-addressable storage system.
4. The method of claim 1 , wherein the first copy instruction includes a token-based copy instruction, and the second copy instruction includes a write via full hash instruction.
5. The method of claim 1 , further comprising:
receiving the second copy instruction at the target system; and
copying each of the one or more data items to the replica of the second volume based on the one or more hash digests that are retrieved from the token-specific snapshot.
6. The method of claim 5 , wherein copying any of the one or more data items includes:
attempting to execute a hash-based write of the data item to the replica of the second volume; and
copying the data item to the replica of the second volume via a full data transfer if the hash-based write fails.
7. A system comprising:
a memory that is configured to store a first volume and a second volume; and
one or more processors that are operatively coupled to the memory, the one or more processors being configured to perform the operations of:
receiving a first copy instruction, the first copy instruction including or referencing a token that represents one or more data items, the token containing an identifier of a token-specific snapshot that is arranged to preserve the one or more data items, the first copy instruction instructing the one or more processors to copy the one or more data items from the first volume to the second volume;
detecting whether each of the first volume and the second volume is currently a source for synchronous replication;
when each of the first volume and the second volume is currently a source for synchronous replication: (i) identifying the token-specific snapshot that corresponds to the token, (ii) retrieving one or more hash digests from the token-specific snapshot, each of the one or more hash digests being associated with a different one of the one or more data items, (iii) transmitting, to a target system, a second copy instruction that instructs the target system to copy the one or more data items to a replica of the second volume that is stored at the target system, the second copy instruction being based on the one or more hash digests, and (iv) copying the one or more data items from the first volume to the second volume; and
when the second volume is not a source for synchronous replication, copying the one or more data items from the first volume to the second volume,
wherein the token-specific snapshot is created in response to a request to create the token for the purpose of preserving the one or more data items that are represented by the token, the token-specific snapshot being created before the first copy instruction is received,
wherein the system and the target system are part of a storage system, and the token-specific snapshot is subject to one or more policies in the storage system that limit access to the token-specific snapshot to accesses that are performed for a purpose of performing token-based operations, and
wherein the storage system is configured to create a snapshot of the first volume that corresponds to unmapped read or write commands, and which is subject to different access restrictions than the token-specific snapshot, the snapshot that corresponds to unmapped read or write commands being different from the token-specific snapshot.
8. The system of claim 7 , the one or more processors are further configured to perform the operation of returning an error, when only the second volume is a source of synchronous replication.
9. A non-transitory computer-readable medium storing one or more processor-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the operations of:
receiving a first copy instruction, the first copy instruction including or referencing a token that represents one or more data items, the token containing an identifier of a token-specific snapshot that is arranged to preserve the one or more data items, the first copy instruction instructing a source system to copy the one or more data items from a first volume to a second volume;
detecting whether the source system is currently performing synchronous replication of the first volume and the second volume;
when the source system is performing synchronous replication of both the first volume and the second volume: (i) identifying the token-specific snapshot, (ii) retrieving one or more hash digests from the token-specific snapshot, each of the one or more hash digests being associated with a different one of the one or more data items, (iii) transmitting, to a target system, a second copy instruction that is associated with the one or more hash digests, the second copy instruction instructing the target system to copy the one or more data items to a replica of the second volume that is stored at the target system, and (iv) copying the one or more data items from the first volume to the second volume; and
when the source system is not performing synchronous replication of the second volume, copying the one or more data items from the first volume to the second volume,
wherein the token-specific snapshot is created in response to a request to create the token for the purpose of preserving the one or more data items that are represented by the token, the token-specific snapshot being created before the first copy instruction is received,
wherein the source system and the target system are part of a storage system, and the token-specific snapshot is subject to one or more policies in the storage system that limit access to the token-specific snapshot to accesses that are performed for a purpose of performing token-based operations, and
wherein the storage system is configured to create a snapshot of the first volume that corresponds to unmapped read or write commands, and which is subject to different access restrictions than the token-specific snapshot, the snapshot that corresponds to unmapped read or write commands being different from the token-specific snapshot.
10. The non-transitory computer-readable medium of claim 9 , wherein the one or more processor-executable instructions, when executed by the one or more processors, further cause the one or more processors to perform the operation of returning an error when only the second volume is a source of synchronous replication.
11. The non-transitory computer-readable medium of claim 9 , wherein the source system and the target system are part of a content-addressable storage system.
12. The non-transitory computer-readable medium of claim 9 , wherein the first copy instruction includes a token-based copy instruction, and the second copy instruction includes a write via full hash instruction.
13. The non-transitory computer-readable medium of claim 9 , wherein the one or more processor-executable instructions, when executed by the one or more processors, further cause the one or more processors to perform the operation of executing the second copy instruction by copying each of the one or more data items to the replica of the second volume based on the one or more hash digests that are retrieved from the token-specific snapshot.
14. The non-transitory computer-readable medium of claim 13 , wherein copying any of the one or more data items includes:
attempting to execute a hash-based write of the data item to the replica of the second volume; and
copying the data item to the replica of the second volume via a full data transfer if the hash-based write fails.
15. The method of claim 1 , wherein the token is included in the first copy instruction.
16. The system of claim 7 , wherein the token is included in the first copy instruction.
17. The non-transitory computer-readable medium of claim 9 , wherein the token is included in the first copy instruction.
18. The method of claim 1 , wherein the token-specific snapshot is identified by retrieving the identifier of the token-specific snapshot from the token.
19. The method of claim 1 , wherein the token-specific snapshot is created contemporaneously with the token.