Provenance-based replication in a storage system

1. A method for implementing provenance-based replication in a storage system, the method comprising:

assigning a globally unique identifier (GUID) to a first snap tree of a first storage array and another GUID to a second snap tree of a second storage array, wherein the first snap tree and the second snap tree are peers of each other with respect to at least one volume replicated between the first storage array and the second storage array;

for each volume (Vj) of a plurality of volumes in the first storage array that is replicated to a corresponding volume (Vi) of a plurality of volumes in the second storage array, assigning a volume pairing identifier (ID) common to both of the volume Vj and the volume Vi;

upon determining data for a volume (V 1 ) of the plurality of volumes in the first storage array has been lost or corrupted:

identifying the peer snap tree from the GUID of the peer snap tree and using the volume pairing ID of the volume V 1 to search the peer snap tree for a volume (V 2 ) of the plurality of volumes in the second storage array, the volume V 2 sharing the volume pairing ID of the volume V 1 ;

retrieving data for the volume V 2 and computing a delta between the data of the volume V 1 and the data of the volume V 2 ; and

reconstructing the lost or corrupted data for the volume V 1 using the delta.

2. The method of claim 1 , further comprising:

upon determining the volume V 1 and the volume V 2 are in sync and are read-only, setting an in-sync attribute to both the volume V 1 and the volume V 2 indicating the volume V 1 and the volume V 2 are currently in sync;

in response to performing a system recovery operation: and

accessing the in-sync attribute for the volume V 1 and the volume V 2 ;

upon determining the in-sync attribute is set for both the volume V 1 and the volume V 2 , bypassing a resynchronization operation on the volume V 1 and the volume V 2 ; and

upon determining the in-sync attribute is not set for at least one of the volume V 1 and the volume V 2 , performing the resynchronization operation on the volume V 1 and the volume V 2 .

3. The method of claim 2 , further comprising:

upon determining, during replication operations, at least one of the volume V 1 and the volume V 2 is out of sync, revoking the in-sync attribute for the volume V 1 and the volume V 2 .

4. The method of claim 2 , further comprising establishing the in-sync attribute for the volume V 1 and the volume V 2 during synchronous replication, the establishing comprising:

copying data from the volume V 1 to the volume V 2 ;

waiting for replication to finish copying the data from the volume V 1 to the volume V 2 ;

for writes received at the volume V 1 from a host after completion of the copying from the volume V 1 to the volume V 2 , replicating the writes from the volume V 1 to the volume V 2 ;

suspending, by the host, further writes to the volume V 1 ;

upon completion of replicating in-flight replication operations from the volume V 1 to the volume V 2 , taking a snapshot of the volume V 1 and taking a snapshot of the volume V 2 ;

providing provenance data to the snapshot of the volume V 1 and the snapshot of the volume V 2 , the provenance data including at least one of the GUID of the first snap tree and the second snap tree, a shared GUID pairing identifier of the first snap tree and the second snap tree, a current time and date, and the in-sync attribute; and

resuming writes from the host to the volume V 1 .

5. The method of claim 4 , wherein the shared GUID pairing identifier indicates that common ancestry data exists between the first snap tree and the second snap tree.

6. The method of claim 4 , further comprising replicating from an immutable volume (W 1 ) in the first storage array to the second storage array, the replicating comprising:

identifying, in the first storage array, the first snap tree corresponding to the volume W 1 ;

searching, in the second storage array, the second snap tree for volumes having common provenance data with provenance data of the volume W 1 ; and

upon determining no common provenance data is found in the second storage array, performing a full copy replication of the volume W 1 .

7. The method of claim 6 , further comprising:

upon determining common provenance is found in the second storage array, searching the first snap tree and the second snap tree for a set of volumes that are in sync with each other;

if no set of volumes are found to be in sync with each other, performing a full copy replication of volume W 1 ;

if the set of volumes are found to be in sync with each other:

taking a snapshot of a volume (W 2 ) off of a volume in the set residing in the second storage array;

establishing a replication relationship from the volume W 1 to the volume W 2 without copying data;

computing a difference set between the volume W 1 and a volume of the set residing in the first storage array;

sending the difference set to the second storage array; and

applying the difference set to the volume W 2 , the applying rendering the volume W 2 and the volume W 1 to be in sync.

8. A system for implementing provenance-based replication, the system comprising:

a memory comprising computer-executable instructions; and

a processor operable by a storage system, the processor executing the computer-executable instructions, the computer-executable instructions when executed by the processor cause the processor to perform operations comprising:

assigning a globally unique identifier (GUID) to a first snap tree of a first storage array and another GUID to a second snap tree of a second storage array, wherein the first snap tree and the second snap tree are peers of each other with respect to at least one volume replicated between the first storage array and the second storage array;

for each volume (Vj) of a plurality of volumes in the first storage array that is replicated to a corresponding volume (Vi) of a plurality of volumes in the second storage array, assigning a volume pairing identifier (ID) common to both of the volume Vj and the volume Vi;

upon determining data for a volume (V 1 ) of the plurality of volumes in the first storage array has been lost or corrupted:

identifying the peer snap tree from the GUID of the peer snap tree and using the volume pairing ID of the volume V 1 to search the peer snap tree for a volume (V 2 ) of the plurality of volumes in the second storage array, the volume V 2 sharing the volume pairing ID of the volume V 1 ;

retrieving data for the volume V 2 and computing a delta between the data of the volume V 1 and the data of the volume V 2 ; and

reconstructing the lost or corrupted data for the volume V 1 using the delta.

9. The system of claim 8 , wherein the operations further comprise:

upon determining the volume V 1 and the volume V 2 are in sync and are read-only, setting an in-sync attribute to both the volume V 1 and the volume V 2 indicating the volume V 1 and the volume V 2 are currently in sync;

in response to performing a system recovery operation: and

accessing the in-sync attribute for the volume V 1 and the volume V 2 ;

upon determining the in-sync attribute is set for both the volume V 1 and the volume V 2 , bypassing a resynchronization operation on the volume V 1 and the volume V 2 ; and

upon determining the in-sync attribute is not set for at least one of the volume V 1 and the volume V 2 , performing the resynchronization operation on the volume V 1 and the volume V 2 .

10. The system of claim 9 , wherein the operations further comprise:

upon determining, during replication operations, at least one of the volume V 1 and the volume V 2 is out of sync, revoking the in-sync attribute for the volume V 1 and the volume V 2 .

11. The system of claim 9 , wherein the operations further comprise establishing the in-sync attribute for the volume V 1 and the volume V 2 during synchronous replication, the establishing comprising:

copying data from the volume V 1 to the volume V 2 ;

waiting for replication to finish copying the data from the volume V 1 to the volume V 2 ;

for writes received at the volume V 1 from a host after completion of the copying from the volume V 1 to the volume V 2 , replicating the writes from the volume V 1 to the volume V 2 ;

suspending, by the host, further writes to the volume V 1 ;

upon completion of replicating in-flight replication operations from the volume V 1 to the volume V 2 , taking a snapshot of the volume V 1 and taking a snapshot of the volume V 2 ;

providing provenance data to the snapshot of the volume V 1 and the snapshot of the volume V 2 , the provenance data including at least one of the GUID of the first snap tree and the second snap tree, a shared GUID pairing identifier of the first snap tree and the second snap tree, a current time and date, and the in-sync attribute; and

resuming writes from the host to the volume V 1 .

12. The system of claim 11 , wherein the shared GUID pairing identifier indicates common ancestry data exists between the first snap tree and the second snap tree.

13. The system of claim 11 , wherein the operations further comprise replicating from an immutable volume (W 1 ) in the first storage array to the second storage array, the replicating comprising:

identifying, in the first storage array, the first snap tree corresponding to the volume W 1 ;

searching, in the second storage array, the second snap tree for volumes having common provenance data with provenance data of the volume W 1 ; and

upon determining no common provenance data is found in the second storage array, performing a full copy replication of the volume W 1 .

14. The system of claim 13 , wherein the operations further comprise:

upon determining common provenance is found in the second storage array, searching the first snap tree and the second snap tree for a set of volumes that are in sync with each other;

if no set of volumes are found to be in sync with each other, performing a full copy replication of volume W 1 ;

if the set of volumes are found to be in sync with each other:

taking a snapshot of a volume (W 2 ) off of a volume in the set residing in the second storage array;

establishing a replication relationship from the volume W 1 to the volume W 2 without copying data;

computing a difference set between the volume W 1 and a volume of the set residing in the first storage array;

sending the difference set to the second storage array; and

applying the difference set to the volume W 2 , the applying rendering the volume W 2 and the volume W 1 to be in sync.

15. A computer program product for implementing provenance-based replication, the computer program product embodied on a non-transitory computer readable medium, the computer program product including instructions that, when executed by a computer, causes the computer to perform operations comprising:

assigning a globally unique identifier (GUID) to a first snap tree of a first storage array and another GUID to a second snap tree of a second storage array, wherein the first snap tree and the second snap tree are peers of each other with respect to at least one volume replicated between the first storage array and the second storage array;

for each volume (Vj) of a plurality of volumes in the first storage array that is replicated to a corresponding volume (Vi) of a plurality of volumes in the second storage array, assigning a volume pairing identifier (ID) common to both of the volume Vj and the volume Vi;

upon determining data for a volume (V 1 ) of the plurality of volumes in the first storage array has been lost or corrupted:

identifying the peer snap tree from the GUID of the peer snap tree and using the volume pairing ID of the volume V 1 to search the peer snap tree for a volume (V 2 ) of the plurality of volumes in the second storage array, the volume V 2 sharing the volume pairing ID of the volume V 1 ;

retrieving data for the volume V 2 and computing a delta between the data of the volume V 1 and the data of the volume V 2 ; and

reconstructing the lost or corrupted data for the volume V 1 using the delta.

16. The computer program product of claim 15 , wherein the operations further comprise:

upon determining the volume V 1 and the volume V 2 are in sync and are read-only, setting an in-sync attribute to both the volume V 1 and the volume V 2 indicating the volume V 1 and the volume V 2 are currently in sync;

in response to performing a system recovery operation: and

accessing the in-sync attribute for the volume V 1 and the volume V 2 ;

upon determining the in-sync attribute is set for both the volume V 1 and the volume V 2 , bypassing a resynchronization operation on the volume V 1 and the volume V 2 ; and

upon determining the in-sync attribute is not set for at least one of the volume V 1 and the volume V 2 , performing the resynchronization operation on the volume V 1 and the volume V 2 .

17. The computer program product of claim 16 , wherein the operations further comprise:

upon determining, during replication operations, at least one of the volume V 1 and the volume V 2 is out of sync, revoking the in-sync attribute for the volume V 1 and the volume V 2 .

18. The computer program product of claim 16 , wherein the operations further comprise establishing the in-sync attribute for the volume V 1 and the volume V 2 during synchronous replication, the establishing comprising:

copying data from the volume V 1 to the volume V 2 ;

waiting for replication to finish copying the data from the volume V 1 to the volume V 2 ;

for writes received at the volume V 1 from a host after completion of the copying from the volume V 1 to the volume V 2 , replicating the writes from the volume V 1 to the volume V 2 ;

suspending, by the host, further writes to the volume V 1 ;

upon completion of replicating in-flight replication operations from the volume V 1 to the volume V 2 , taking a snapshot of the volume V 1 and taking a snapshot of the volume V 2 ;

providing provenance data to the snapshot of the volume V 1 and the snapshot of the volume V 2 , the provenance data including at least one of the GUID of the first snap tree and the second snap tree, a shared GUID pairing identifier of the first snap tree and the second snap tree, a current time and date, and the in-sync attribute; and

resuming writes from the host to the volume V 1 .

19. The computer program product of claim 18 , wherein the shared GUID pairing identifier indicates common ancestry data exists between the first snap tree and the second snap tree.

20. The computer program product of claim 18 , wherein the operations further comprise replicating from an immutable volume (W 1 ) in the first storage array to the second storage array, the replicating comprising:

identifying, in the first storage array, the first snap tree corresponding to the volume W 1 ;

searching, in the second storage array, the second snap tree for volumes having common provenance data with provenance data of the volume W 1 ; and

upon determining no common provenance data is found in the second storage array, performing a full copy replication of the volume W 1 ;

upon determining common provenance is found in the second storage array, searching the first snap tree and the second snap tree for a set of volumes that are in sync with each other;

if no set of volumes are found to be in sync with each other, performing a full copy replication of volume W 1 ;

if the set of volumes are found to be in sync with each other:

taking a snapshot of a volume (W 2 ) off of a volume in the set residing in the second storage array;

establishing a replication relationship from the volume W 1 to the volume W 2 without copying data;

computing a difference set between the volume W 1 and a volume of the set residing in the first storage array;

sending the difference set to the second storage array; and

applying the difference set to the volume W 2 , the applying rendering the volume W 2 and the volume W 1 to be in sync.