Rolling cache configuration for a data replication system
View Patent ↗Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.
1. A system for facilitating data synchronization following a network failure in a data replication environment, the system comprising:
a plurality of log entries representing data management operations generated by an application on a source computer for data stored on a first storage device, each of the plurality of log entries including a unique identifier, wherein one or more of the plurality of log entries identify at least one known good state of the application;
a cache memory configured to store a portion of the plurality of log entries such that when a size of the portion of log entries exceeds a storage threshold of the cache memory, the cache memory deletes one or more of the least recently received log entries until the size is less than the storage threshold;
at least one destination computer configured to process the plurality of log entries to replicate data to a second storage device, the at least one destination computer being further configured to record the unique identifier of a most recently processed log entry, wherein the destination computer processes the log entries to create replicated data that replicates the data stored on the first storage device, the replicated data on the destination computer being a recoverable copy at the known good state of the application on the source computer; and
at least one replication routine configured to sequentially retrieve each of the plurality of log entries from the cache memory and to forward each log entry to the at least one destination computer without waiting for an acknowledgement from the at least one destination computer;
wherein when the destination computer needs to resynchronize, a replication thread uses the unique identifier of the most recently processed log entry recorded on the destination computer, to locate a log entry in the cache memory that occurs after the most recently processed log entry; and
wherein when the cache memory no longer has a copy of the log entry that occurs after the most recently processed log entry, a mirror operation is performed to synchronize the data stored on the source computer with the data stored on the destination computer.
2. The system of claim 1 , wherein the unique identifier comprises a sequence number.
3. The system of claim 1 , wherein the cache memory comprises a permanent disk cache.
4. The system of claim 1 , wherein the cache memory has a storage threshold of between approximately five gigabytes and approximately twenty gigabytes.
5. The system of claim 1 , wherein the at least one destination computer comprises a plurality of destination computers.
6. The system of claim 5 , wherein the at least one replication routine comprises a plurality of replication threads.
7. The system of claim 6 , wherein each of the plurality of replication threads communicates with one of the plurality of destination computers.
8. The system of claim 1 , wherein the at least one replication routine does not store a copy of transmitted log entries.
9. The system of claim 1 , wherein, following a network failure, the at least one replication routine is configured to:
receive from the at least one destination computer the unique identifier of the most recently processed log entry;
locate a copy of the most recently processed log entry in the cache memory; and
forward copies of log entries following the most recently processed log entry to the at least one destination computer.