Sharing memory resources between asynchronous replication workloads
Techniques are provided for managing the utilization of shared memory resources among a plurality of asynchronous replication workloads. A data replication system maintains a respective replication journal for each replication workload in an allocated portion of system memory of a source server node. Each replication journal is divided into journal barrier intervals by respective individual barriers for transmission to a destination server node. The data replication system waits for a triggering event to occur for generating a common barrier, wherein the triggering event is configured to prevent exhaustion of the allocated portion of system memory for storing the replication journals. The common barrier is generated to close the replication journals of some or all of the replication workloads in advance of the respective individual barriers associated, and the replication journals closed by the common barrier are transmitted to the destination server node.
1. A method, comprising:
receiving, by a data replication system, input/output (I/O) write requests associated with each of a plurality of replication workloads executing concurrently on a source server node which hosts the data replication system;
maintaining, by the data replication system, a respective replication journal for each replication workload in an allocated portion of system memory of the source server node, wherein each replication journal is divided into journal barrier intervals by respective individual barriers for transmission to a destination server node, wherein the journal barrier intervals of the respective replication journals comprise journal data collection intervals for the respective replication workloads, and wherein the journal data collection intervals are based, at least in part, on respective recovery point objectives associated with the replication workloads;
determining, by the data replication system, an occurrence of a triggering event for generating a common barrier, wherein the triggering event is configured to prevent exhaustion of the allocated portion of system memory for storing the replication journals of the plurality of replication workloads in the allocated portion of system memory of the source server node;
generating, by the data replication system, the common barrier to prematurely close the journal data collection intervals of the respective replication journals of at least two replication workloads in response to the occurrence of the triggering event; and
transmitting, by the data replication system, the replication journals closed by the common barrier, to the destination server node.
2. The method of claim 1 , wherein the triggering event comprises an expiration of a specified common barrier interval.
3. The method of claim 2 , wherein the specified common barrier interval is determined as a function of (i) a total size of the system memory, (ii) a maximum write throughput of the data replication system, and (iii) a size of memory for processing a single I/O write request.
4. The method of claim 2 , wherein the specified common barrier interval is determined as a function of (i) a total size of the allocated portion of system memory, (ii) a maximum I/O operations per second of the data replication system, and (iii) a size of memory for processing a single I/O write request.
5. The method of claim 1 , wherein the triggering event comprises a number of I/O write requests of the plurality of replication workloads meeting a threshold number of I/O requests from a time of generation of a previous common barrier.
6. The method of claim 1 , wherein the triggering event comprises a number of I/O write requests of the plurality of replication workloads meeting a threshold number of I/O requests before expiration of a specified common barrier interval.
7. The method of claim 1 , further comprising excluding a given replication journal for a given one of the replication workloads from being closed by the common barrier, when the journal barrier interval of the given replication journal is deemed to have been recently started by a predetermined amount of time prior to a time of the common barrier.
8. The method of claim 1 , further comprising excluding a given replication journal for a given replication workload from being closed by the common barrier, when the journal barrier interval of the given replication journal is deemed to have utilized an amount of system memory which is below a target threshold memory usage at a time of the common barrier.
9. An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code is executable by one or more processors to implement a method which comprises:
receiving, by a data replication system, input/output (I/O) write requests associated with each of a plurality of replication workloads executing concurrently on a source server node which hosts the data replication system;
maintaining, by the data replication system, a respective replication journal for each replication workload in an allocated portion of system memory of the source server node, wherein each replication journal is divided into journal barrier intervals by respective individual barriers for transmission to a destination server node, wherein the journal barrier intervals of the respective replication journals comprise journal data collection intervals for the respective replication workloads, and wherein the journal data collection intervals are based, at least in part, on respective recovery point objectives associated with the replication workloads;
determining, by the data replication system, an occurrence of a triggering event for generating a common barrier, wherein the triggering event is configured to prevent exhaustion of the allocated portion of system memory for storing the replication journals of the plurality of replication workloads in the allocated portion of system memory of the source server node;
generating, by the data replication system, the common barrier to prematurely close the journal data collection intervals of the respective replication journals of at least two replication workloads in response to the occurrence of the triggering event; and
transmitting, by the data replication system, the replication journals closed by the common barrier, to the destination server node.
10. The article of manufacture of claim 9 , wherein the triggering event comprises an expiration of a specified common barrier interval.
11. The article of manufacture of claim 10 , wherein the specified common barrier interval is determined as a function of (i) a total size of the system memory, (ii) a maximum write throughput of the data replication system, and (iii) a size of memory for processing a single I/O write request.
12. The article of manufacture of claim 10 , wherein the specified common barrier interval is determined as a function of (i) a total size of the system memory, (ii) a maximum I/O operations per second of the data replication system, and (iii) a size of memory for processing a single I/O write request.
13. The article of manufacture of claim 9 , wherein the triggering event comprises a number of I/O write requests of the plurality of replication workloads meeting a threshold number of I/O requests from a time of generation of a previous common barrier.
14. The article of manufacture of claim 9 , wherein the triggering event comprises a number of I/O write requests of the plurality of replication workloads meeting a threshold number of I/O requests before expiration of a specified common barrier interval.
15. The article of manufacture of claim 9 , further comprising program code for excluding a given replication journal for a given one of the replication workloads from being closed by the common barrier, when the journal barrier interval of the given replication journal is deemed to have been recently started by a predetermined amount of time prior to a time of the common barrier.
16. The article of manufacture of claim 9 , further comprising program code for excluding a given replication journal for a given replication workload from being closed by the common barrier, when the journal barrier interval of the given replication journal is deemed to have utilized an amount of system memory which is below a target threshold memory usage at a time of the common barrier.
17. A server node, comprising:
at least one processor; and
a system memory configured to store program code, wherein the program code is executable by the at least one processor to implement a data replication system which is configured to:
receive input/output (I/O) write requests associated with each of a plurality of replication workloads executing concurrently on the server node;
maintain a respective replication journal for each replication workload in an allocated portion of the system memory of the server node, wherein each replication journal is divided into journal barrier intervals by respective individual barriers for transmission to a destination server node, wherein the journal barrier intervals of the respective replication journals comprise journal data collection intervals for the respective replication workloads, and wherein the journal data collection intervals are based, at least in part, on respective recovery point objectives associated with the replication workloads;
determine an occurrence of a triggering event for generating a common barrier, wherein the triggering event is configured to prevent exhaustion of the allocated portion of system memory for storing the replication journals of the plurality of replication workloads in the allocated portion of system memory of the source server node;
generate the common barrier to prematurely close the journal data collection intervals of the respective replication journals of at least two replication workloads in response to the occurrence of the triggering event; and
transmit the replication journals closed by the common barrier, to the destination server node.
18. The server node of claim 17 , wherein the triggering event comprises an expiration of a specified common barrier interval, and wherein the specified common barrier interval is determined as a function of (i) a total size of the system memory, (ii) a size of memory for processing a single I/O write request, and (iii) one of a maximum write throughput of the data replication system, and a maximum I/O operations per second of the data replication system.
19. The server node of claim 17 , wherein the triggering event comprises one of (i) a number of I/O write requests of the plurality of replication workload meeting a threshold number of I/O requests from a time of generation of a previous common barrier and (ii) a number of I/O write requests of the plurality of replication workload meeting a threshold number of I/O requests before expiration of a specified common barrier interval.
20. The server node of claim 17 , wherein the data replication system is further configured to at least one of (i) exclude a given replication journal for a given one of the replication workloads from being closed by the common barrier, when the journal barrier interval of the given replication journal is deemed to have been recently started by a predetermined amount of time prior to a time of the common barrier, and (ii) exclude a given replication journal for a given replication workload from being closed by the common barrier, when the journal barrier interval of the given replication journal is deemed to have utilized an amount of system memory which is below a target threshold memory usage at a time of the common barrier.