Granted Patent B2

US 11,513,900 · App. 17/237,100 · Granted Nov 29, 2022

Inventors: Bhaskar Bora (Shrewsbury, MA); Mark J. Halstead (Holliston, MA); Jeffrey Wilson (Franklin, MA)

Assignee: EMC IP Holding Company LLC

G06F11/1451G06F3/065G06F3/067G06F3/0619G06F3/0683G06F11/1435G06F11/1469

Abstract

Claims (75)

1. For a system including a first storage system, a second storage system, and a first logical storage unit, wherein the first storage system is configured to replicate data of the first logical storage unit to a second logical storage unit on the second storage system, a method comprising:

recording a list of a plurality of snapshots of the first logical storage unit that were taken on the first storage system but not replicated to the second storage system because of a first period of time during which the replication of the first logical storage unit to the second logical storage unit is inactive; and

in response to the replication of the first logical storage unit to the second logical storage unit becoming active, accessing the list and replicating the plurality of snapshots of the list from the first storage system to replica snapshots on the second storage system,

wherein the replicating of the plurality of snapshots is performed concurrently to replicating write operations as part of the replication of the first logical storage unit to the second logical storage unit.

2. The method of claim 1 , wherein the replicating of the plurality of snapshots is performed as part of a first process that is executed independently of one or more processes executing the replicating of the write operations.

3. The method of claim 1 , wherein the plurality of snapshots are listed in the list in a chronological order from an earliest point in time corresponding to a snapshot to a latest point in time corresponding to a snapshot, and wherein the replicating of the plurality of snapshots includes:

replicating a first snapshot of the plurality of snapshots to a first replica snapshot on the second storage system, including, for each logical storage element of the first logical storage unit:

determining if one or more write operations received on the first storage system for the logical storage element have had data stored on the first storage system but not yet replicated to the second logical storage unit on the second storage system; and

if the one or more write operations for the logical storage element have had data stored on the first storage system but not yet replicated to the second logical storage unit, waiting for the data of the one or more write operations to be replicated to the second logical storage unit before replicating snapshot data of the logical storage element from the first storage system to the first replica snapshot on the second storage system.

4. The method of claim 3 , wherein the replicating of the first snapshot to the first replica snapshot includes, for each logical storage element of the first logical storage unit:

determining if any snapshot data for the logical storage element has been stored on the first storage system, but not yet replicated to the second storage system; and

if any snapshot data for the logical storage element has been stored on the first storage system, but not yet replicated to the second storage system, replicating the snapshot data from first storage system to the second storage system.

5. The method of claim 1 , wherein the plurality of snapshots are listed in the list in a chronological order from an earliest point in time corresponding to a snapshot to a latest point in time corresponding to a snapshot, and wherein the replicating of the plurality of snapshots includes:

after replicating a first snapshot of the plurality of snapshots, replicating a second snapshot of the plurality of snapshots to a second replica snapshot on the second storage system, including:

for each logical storage element of the first logical storage unit, determining if the logical storage element data has been modified between the first snapshot and the second snapshot; and

performing the remaining processing for replicating the second snapshot snapshots to the second replica snapshot on only the logical storage elements of the first logical storage unit that have been modified between the first snapshot and the second snapshot.

6. The method of claim 5 , further comprising, for each logical storage element of the first logical storage unit that has been modified between the first snapshot and the second snapshot:

determining if a value of the logical storage element for the second snapshot is a same value as a current value of the logical storage element on the first storage system; and

if the value of the logical storage element for the second snapshot is the same value as the current value of the logical storage element on the first storage system, updating metadata of the second replica snapshot for the logical storage element to reference the current value of the logical storage element on the second storage system.

7. The method of claim 5 , further comprising, for each logical storage element of the first logical storage unit that has been modified between the first snapshot and the second snapshot:

determining if a value of the logical storage element for the second snapshot is a same value as a current value of the logical storage element on the first storage system; and

if the value of the logical storage element for the second snapshot is not the same value as the current value of the logical storage element on the first storage system:

copying snapshot data of the logical storage element specified for the second snapshot from the first storage system to the second storage system; and

updating metadata of the second replica snapshot for the logical storage element to reference the copied snapshot data.

8. A system comprising:

a first storage system;

a second storage system;

a first logical storage unit, wherein the first storage system is configured to replicate data of the first logical storage unit to a second logical storage unit on the second storage system; and

executable logic that implements a method including:

recording a list of a plurality of snapshots of the first logical storage unit that were taken on the first storage system but not replicated to the second storage system because of a first period of time during which the replication of the first logical storage unit to the second logical storage unit is inactive; and

in response to the replication of the first logical storage unit to the second logical storage unit becoming active, accessing the list and replicating the plurality of snapshots of the list from the first storage system to replica snapshots on the second storage system,

wherein the replicating of the plurality of snapshots is performed concurrently to replicating write operations as part of the replication of the first logical storage unit to the second logical storage unit.

9. The system of claim 8 , wherein the replicating of the plurality of snapshots is performed as part of a first process that is executed independently of one or more processes executing the replicating of the write operations.

10. The system of claim 8 , wherein the plurality of snapshots are listed in the list in a chronological order from an earliest point in time corresponding to a snapshot to a latest point in time corresponding to a snapshot, and wherein the replicating of the plurality of snapshots includes:

replicating a first snapshot of the plurality of snapshots to a first replica snapshot on the second storage system, including, for each logical storage element of the first logical storage unit:

determining if one or more write operations received on the first storage system for the logical storage element have had data stored on the first storage system but not yet replicated to the second logical storage unit on the second storage system; and

if the one or more write operations for the logical storage element have had data stored on the first storage system but not yet replicated to the second logical storage unit, waiting for the data of the one or more write operations to be replicated to the second logical storage unit before replicating snapshot data of the logical storage element from the first storage system to the first replica snapshot on the second storage system.

11. The system of claim 10 , wherein the replicating of the first snapshot to the first replica snapshot includes, for each logical storage element of the first logical storage unit:

determining if any snapshot data for the logical storage element has been stored on the first storage system, but not yet replicated to the second storage system; and

if any snapshot data for the logical storage element has been stored on the first storage system, but not yet replicated to the second storage system, replicating the snapshot data from first storage system to the second storage system.

12. The system of claim 8 , wherein the plurality of snapshots are listed in the list in a chronological order from an earliest point in time corresponding to a snapshot to a latest point in time corresponding to a snapshot, and wherein the replicating of the plurality of snapshots includes:

after replicating a first snapshot of the plurality of snapshots, replicating a second snapshot of the plurality of snapshots to a second replica snapshot on the second storage system, including:

for each logical storage element of the first logical storage unit, determining if the logical storage element data has been modified between the first snapshot and the second snapshot; and

performing the remaining processing for replicating the second snapshot snapshots to the second replica snapshot on only the logical storage elements of the first logical storage unit that have been modified between the first snapshot and the second snapshot.

13. The system of claim 12 , wherein the method further includes, for each logical storage element of the first logical storage unit that has been modified between the first snapshot and the second snapshot:

determining if a value of the logical storage element for the second snapshot is a same value as a current value of the logical storage element on the first storage system; and

if the value of the logical storage element for the second snapshot is the same value as the current value of the logical storage element on the first storage system, updating metadata of the second replica snapshot for the logical storage element to reference the current value of the logical storage element on the second storage system.

14. The system of claim 12 , wherein the method further includes, for each logical storage element of the first logical storage unit that has been modified between the first snapshot and the second snapshot:

determining if a value of the logical storage element for the second snapshot is a same value as a current value of the logical storage element on the first storage system; and

if the value of the logical storage element for the second snapshot is not the same value as the current value of the logical storage element on the first storage system:

copying snapshot data of the logical storage element specified for the second snapshot from the first storage system to the second storage system; and

updating metadata of the second replica snapshot for the logical storage element to reference the copied snapshot data.

15. For a system including a first storage system, a second storage system, and a first logical storage unit, wherein the first storage system is configured to replicate data of the first logical storage unit to a second logical storage unit on the second storage system, computer-readable media having software stored thereon comprising:

executable code that controls, recording a list of a plurality of snapshots of the first logical storage unit that were taken on the first storage system but not replicated to the second storage system because of a first period of time during which the replication of the first logical storage unit to the second logical storage unit is inactive; and

executable code that controls, in response to the replication of the first logical storage unit to the second logical storage unit becoming active, accessing the list and replicating the plurality of snapshots of the list from the first storage system to replica snapshots on the second storage system,

wherein the replicating of the plurality of snapshots is performed concurrently to replicating write operations as part of the replication of the first logical storage unit to the second logical storage unit.

16. The computer-readable media of claim 15 , wherein the plurality of snapshots are listed in the list in a chronological order from an earliest point in time corresponding to a snapshot to a latest point in time corresponding to a snapshot, and wherein the replicating of the plurality of snapshots includes:

replicating a first snapshot of the plurality of snapshots to a first replica snapshot on the second storage system, including, for each logical storage element of the first logical storage unit:

determining if one or more write operations received on the first storage system for the logical storage element have had data stored on the first storage system but not yet replicated to the second logical storage unit on the second storage system; and

if the one or more write operations for the logical storage element have had data stored on the first storage system but not yet replicated to the second logical storage unit, waiting for the data of the one or more write operations to be replicated to the second logical storage unit before replicating snapshot data of the logical storage element from the first storage system to the first replica snapshot on the second storage system.

17. The computer-readable media of claim 16 , wherein the replicating of the first snapshot to the first replica snapshot includes, for each logical storage element of the first logical storage unit:

determining if any snapshot data for the logical storage element has been stored on the first storage system, but not yet replicated to the second storage system; and

if any snapshot data for the logical storage element has been stored on the first storage system, but not yet replicated to the second storage system, replicating the snapshot data from first storage system to the second storage system.

18. The computer-readable media of claim 15 , wherein the plurality of snapshots are listed in the list in a chronological order from an earliest point in time corresponding to a snapshot to a latest point in time corresponding to a snapshot, and wherein the replicating of the plurality of snapshots includes:

after replicating a first snapshot of the plurality of snapshots, replicating a second snapshot of the plurality of snapshots to a second replica snapshot on the second storage system, including:

for each logical storage element of the first logical storage unit, determining if the logical storage element data has been modified between the first snapshot and the second snapshot; and

performing the remaining processing for replicating the second snapshot snapshots to the second replica snapshot on only the logical storage elements of the first logical storage unit that have been modified between the first snapshot and the second snapshot.

19. The computer-readable media of claim 18 , wherein the software further comprises, for each logical storage element of the first logical storage unit that has been modified between the first snapshot and the second snapshot:

executable code that controls determining if a value of the logical storage element for the second snapshot is a same value as a current value of the logical storage element on the first storage system; and

executable code that controls, if the value of the logical storage element for the second snapshot is the same value as the current value of the logical storage element on the first storage system, updating metadata of the second replica snapshot for the logical storage element to reference the current value of the logical storage element on the second storage system.

20. The computer-readable media of claim 18 , wherein the software further comprises, for each logical storage element of the first logical storage unit that has been modified between the first snapshot and the second snapshot:

executable code that controls determining if a value of the logical storage element for the second snapshot is a same value as a current value of the logical storage element on the first storage system; and

executable code that controls, if the value of the logical storage element for the second snapshot is not the same value as the current value of the logical storage element on the first storage system:

copying snapshot data of the logical storage element specified for the second snapshot from the first storage system to the second storage system; and

updating metadata of the second replica snapshot for the logical storage element to reference the copied snapshot data.

Assignments (10)