Granted Patent B2

US 11,494,405 · App. 17/128,419 · Granted Nov 8, 2022

Inventors: Xiangping Chen (Sherborn, MA); David Meiri (Somerville, MA)

Assignee: EMC IP Holding Company LLC

G06F16/27G06F9/524G06F11/1464G06F16/2343G06F16/2365G06F2201/805

Abstract

Claims (59)

1. An apparatus comprising:

at least one processing device comprising a processor coupled to a memory;

said at least one processing device being configured:

to maintain a replication journal for recording replication write requests in a storage system;

to detect a failure impacting the replication journal;

to initiate recovery of the replication journal responsive to the detected failure;

in conjunction with the recovery of the replication journal, to maintain a lock contention table that characterizes lock contentions between address lock ranges required for the recovery of the replication journal and address lock ranges required by other write requests in the storage system; and

to utilize the lock contention table to resolve one or more potential deadlocks that would otherwise prevent completion of the recovery of the replication journal;

wherein a given entry of the replication journal is removed therefrom based at least in part on completion of replication processing for a corresponding one of the replication write requests in an additional storage system.

2. The apparatus of claim 1 wherein said at least one processing device comprises at least a portion of a storage controller of a first storage system, the first storage system being configured to participate in a replication process with a second storage system, and wherein the replication write requests are generated as part of the replication process.

3. The apparatus of claim 2 wherein the first and second storage systems are arranged in an active-active configuration relative to one another for performance of the replication process.

4. The apparatus of claim 1 wherein said at least one processing device comprises a particular one of a plurality of storage nodes of a distributed storage system, each such storage node comprising a set of processing modules configured to communicate with corresponding sets of processing modules on other ones of the storage nodes, the sets of processing modules of the storage nodes of the distributed storage system collectively comprising at least a portion of a distributed storage controller of the distributed storage system.

5. The apparatus of claim 4 wherein different instances of the lock contention table are maintained by different ones of the processing modules of the sets of processing modules of the respective storage nodes of the distributed storage system.

6. The apparatus of claim 1 wherein maintaining the replication journal comprises:

creating a journal entry in the replication journal for a given one of the replication write requests responsive to data of the write request being persisted in a first storage system that receives the write request from a host device; and

removing the journal entry in the replication journal for the given one of the replication write requests responsive to data of the write request being persisted in a second storage system that receives the data from the first storage system.

7. The apparatus of claim 1 wherein initiating recovery of the replication journal comprises:

acquiring address locks needed to apply journal entries of the replication journal; and

performing a replication journal recovery operation by sending a replication write request associated with a selected one of the journal entries from a first storage system to a second storage system.

8. The apparatus of claim 7 wherein responsive to the detected failure, normal processing of other write requests is temporarily interrupted until the address locks needed to apply journal entries of the replication journal are acquired.

9. The apparatus of claim 7 wherein maintaining a lock contention table comprises:

determining whether or not the second storage system rejects the replication write request due to a lock collision; and

responsive to an affirmative determination, releasing a corresponding lock and updating the lock contention table to include a table entry comprising a lock contention range and journal entry information for the selected journal entry.

10. The apparatus of claim 9 wherein the journal entry information associated with the lock contention range in the lock contention table comprises at least an offset and a length for the corresponding replication write request.

11. The apparatus of claim 9 wherein performing a replication journal recovery operation by sending a replication write request associated with a selected one of the journal entries, determining whether or not the second storage system rejects the replication write request due to a lock collision, and responsive to an affirmative determination, releasing the corresponding lock and updating the lock contention table to include a table entry comprising a lock contention range and journal entry information for the selected journal entry, are repeated for each of one or more additional selected journal entries in the replication journal.

12. The apparatus of claim 11 wherein the repeating continues until all journal entries have been selected and processed and the lock contention table is empty.

13. The apparatus of claim 1 wherein utilizing the lock contention table to resolve one or more potential deadlocks that would otherwise prevent completion of the recovery of the replication journal comprises:

for a given one of the other write requests, determining whether or not it has a target data range that at least partially matches a lock contention range in a table entry of the lock contention table;

responsive to the given other write request having a target data range that at least partially matches a lock contention range in a table entry of the lock contention table, checking a data range in a corresponding journal entry of the replication journal;

responsive to a full match between the target data range and the data range in the corresponding journal entry, clearing both the table entry for that lock contention range and the corresponding journal entry; and

responsive to a partial match between the target data range and the data range in the corresponding journal entry, modifying the corresponding journal entry to include only a non-matching portion of the data range.

14. The apparatus of claim 1 wherein utilizing the lock contention table to resolve one or more potential deadlocks that would otherwise prevent completion of the recovery of the replication journal comprises selectively aborting portions of the recovery of the replication journal based at least in part on one or more table entries of the lock contention table in a manner that maintains write order consistency and cross-site consistency for first and second storage systems participating in an active-active replication process.

15. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes said at least one processing device:

maintaining a replication journal for recording replication write requests in a storage system;

detecting a failure impacting the replication journal;

initiating recovery of the replication journal responsive to the detected failure;

in conjunction with the recovery of the replication journal, maintaining a lock contention table that characterizes lock contentions between address lock ranges required for the recovery of the replication journal and address lock ranges required by other write requests in the storage system; and

utilizing the lock contention table to resolve one or more potential deadlocks that would otherwise prevent completion of the recovery of the replication journal;

wherein a given entry of the replication journal is removed therefrom based at least in part on completion of replication processing for a corresponding one of the replication write requests in an additional storage system.

16. The computer program product of claim 15 wherein utilizing the lock contention table to resolve one or more potential deadlocks that would otherwise prevent completion of the recovery of the replication journal comprises:

for a given one of the other write requests, determining whether or not it has a target data range that at least partially matches a lock contention range in a table entry of the lock contention table;

responsive to the given other write request having a target data range that at least partially matches a lock contention range in a table entry of the lock contention table, checking a data range in a corresponding journal entry of the replication journal;

responsive to a full match between the target data range and the data range in the corresponding journal entry, clearing both the table entry for that lock contention range and the corresponding journal entry; and

responsive to a partial match between the target data range and the data range in the corresponding journal entry, modifying the corresponding journal entry to include only a non-matching portion of the data range.

17. The computer program product of claim 15 wherein utilizing the lock contention table to resolve one or more potential deadlocks that would otherwise prevent completion of the recovery of the replication journal comprises selectively aborting portions of the recovery of the replication journal based at least in part on one or more table entries of the lock contention table in a manner that maintains write order consistency and cross-site consistency for first and second storage systems participating in an active-active replication process.

18. A method comprising:

to maintain a replication journal for recording replication write requests in a storage system;

to detect a failure impacting the replication journal;

to initiate recovery of the replication journal responsive to the detected failure;

in conjunction with the recovery of the replication journal, to maintain a lock contention table that characterizes lock contentions between address lock ranges required for the recovery of the replication journal and address lock ranges required by other write requests in the storage system; and

to utilize the lock contention table to resolve one or more potential deadlocks that would otherwise prevent completion of the recovery of the replication journal;

wherein a given entry of the replication journal is removed therefrom based at least in part on completion of replication processing for a corresponding one of the replication write requests in an additional storage system; and

wherein the method is implemented by at least one processing device comprising a processor coupled to a memory.

19. The method of claim 18 wherein utilizing the lock contention table to resolve one or more potential deadlocks that would otherwise prevent completion of the recovery of the replication journal comprises:

for a given one of the other write requests, determining whether or not it has a target data range that at least partially matches a lock contention range in a table entry of the lock contention table;

responsive to the given other write request having a target data range that at least partially matches a lock contention range in a table entry of the lock contention table, checking a data range in a corresponding journal entry of the replication journal;

responsive to a full match between the target data range and the data range in the corresponding journal entry, clearing both the table entry for that lock contention range and the corresponding journal entry; and

responsive to a partial match between the target data range and the data range in the corresponding journal entry, modifying the corresponding journal entry to include only a non-matching portion of the data range.

20. The method of claim 18 wherein utilizing the lock contention table to resolve one or more potential deadlocks that would otherwise prevent completion of the recovery of the replication journal comprises selectively aborting portions of the recovery of the replication journal based at least in part on one or more table entries of the lock contention table in a manner that maintains write order consistency and cross-site consistency for first and second storage systems participating in an active-active replication process.

Assignments (9)

Continuity (1)

Cited By (1)

US 12,393,366