Resynchronization of replicated data
Mechanisms are provided for efficient resynchronization of replicated data. A hash value is generated for a chunk of data replicated from a source node to a target node. The chunk of data may be a file deduplicated and compressed at both a source node and a target node. A current sequence number is determined and a sequence number and hash tuple is maintained for the chunk of data at both the source node and the target node. Sequence numbers are modified whenever the data is modified. Current sequence numbers and sequence number and hash values in the sequence number hash tuples at the source node and the target node may be compared to determine whether data is still synchronized at a later point in time or whether data requires resynchronization.
1. A method, comprising:
determining an initial data tuple for a chunk of data during replication of the chunk of data from a source node to a target node, the initial data tuple comprising an initial sequence number and an initial hash value, the initial sequence number indicating a number of times the chunk of data has been modified, the initial hash value comprising the result of applying a hash function to the chunk of data;
transmitting the initial data tuple for the chunk of data to the target node;
maintaining the initial data tuple at both the source node and the target node thereby forming a maintained source data tuple and a maintained target data tuple, wherein the maintained source data tuple includes a maintained source hash value and a maintained source sequence number corresponding to the initial sequence number, and wherein the maintained target data tuple includes a maintained target hash value and a maintained target sequence number corresponding to the initial sequence number;
receiving a request to determine whether data corresponding to the chunk at the source node is synchronized with the data corresponding to the chunk at the target node, the request corresponding to the maintained source data tuple and the maintained target data tuple;
determining whether the data at the source node is synchronized with the data at the target node by comparing a current source sequence number, separate from the maintained source sequence number, at the source node to the maintained source sequence number at the source node to indicate whether the data chunk has been modified at the source node, by comparing a current target sequence number, separate from the maintained target sequence number, at the target node to the maintained target sequence number at the target node to determine whether the data chunk has been modified at the target node, and by comparing the maintained source data tuple at the source node to the maintained target data tuple at the target node to determine if the data chunk is out-of-sync, wherein the current source sequence number is separate from the current target sequence number; and
performing resynchronization if the data at the source node is determined to not be synchronized with the data at the target node.
2. The method of claim 1 , wherein the source node is an active node and the target node is a standby node.
3. The method of claim 1 , wherein the initial sequence number and the initial hash value are maintained in extended attributes associated with the chunk.
4. The method of claim 3 , wherein the chunk is a file.
5. The method of claim 1 , wherein if any of the sequence numbers do not match, the data is not synchronized and a resynchronization operation is applied.
6. The method of claim 1 , wherein if the current source and maintained source sequence numbers do match or if the current target and maintained target sequence numbers do match, it is determined whether the maintained source hash value is equal to the maintained target hash value.
7. The method of claim 6 , wherein if the maintained source and maintained target hash values do not match, the data is not synchronized and a resynchronization operation is applied.
8. The method of claim 6 , wherein if the maintained source and maintained target hash values do match, it is determined whether the maintained source sequence number is equal to the maintained target sequence number.
9. The method of claim 8 , wherein if any of the aforementioned sequence numbers do not match, the data is not synchronized and a resynchronization operation is applied.
10. The method of claim 8 , wherein if all of the aforementioned sequence numbers and all of the aforementioned hash values do match, the data is in sync and no resynchronization is required.
11. The method of claim 1 , wherein data is determined to be synchronized by using the following algorithm:
is_data_insync( )
begin
if (primary.hashtuple.seqnum==primary.current_seqnum &&secondary.hashtuple.seqnum==secondary.current_seqnum && primary.hashtuple.hash==secondary.hashtuple.hash && primary.hashtuple.seqnum==secondary.hashtuple.seqnum)
begin return in-sync;
end
return out-of-sync;
end.
12. The method of claim 1 , wherein the maintained source sequence number is stored in a datastore suitcase.
13. The method of claim 12 , wherein the datastore suitcase includes an index portion and a data portion.
14. The method of claim 13 , wherein the index portion includes indices, data offsets, and data reference counts.
15. The method of claim 14 , wherein the data portion includes indices, data, and last file references.
16. A source node, comprising:
a processor configured to determine an initial data tuple for a chunk of data during replication of the chunk of data from a source node to a target node, the initial data tuple comprising an initial sequence number and an initial hash value, the initial sequence number indicating a number of times the chunk of data has been modified, the initial hash value comprising the result of applying a hash function to the chunk of data;
an interface configured to transmit the initial data tuple for the chunk of data to the target node;
memory configured to maintain the initial data tuple thereby forming a maintained source data tuple, wherein the maintained source data tuple includes a maintained source hash value and a maintained source sequence number corresponding to the initial sequence number;
wherein the interface is operable to receive request to determine whether data corresponding to the chunk at the source node is synchronized with the data corresponding to the chunk at the target node, wherein the request corresponds to the maintained source data tuple and the maintained target data tuple, wherein the request indicates that a current target sequence number at the target node has been compared to a maintained target sequence number at the target node to determine whether the data chunk has been modified at the target node, and wherein the processor is operable to determine whether the data at the source node is synchronized with the data at the target node by comparing a current source sequence number at the source node to the maintained source sequence number at the source node to indicate whether the data chunk has been modified at the source node and by comparing the maintained source data tuple at the source node to a maintained target data tuple at the target node to determine if the data chunk is out-of-sync, wherein the current target sequence number is separate from the current source sequence number; and
wherein the processor is further operable to perform resynchronization if the data at the source node is determined to not be synchronized with the data at the target node.
17. The source node of claim 16 , wherein the source node is an active node and the target node is a standby node.
18. The source node of claim 16 , wherein the initial sequence number and the initial hash value are maintained in extended attributes associated with the chunk.
19. The source node of claim 18 , wherein the chunk is a file.
20. A non-transitory computer readable storage medium, comprising:
computer code for determining an initial data tuple for a chunk of data during replication of the chunk of data from a source node to a target node, the initial data tuple comprising an initial sequence number and an initial hash value, the initial sequence number indicating a number of times the chunk of data has been modified, the initial hash value comprising the result of applying a hash function to the chunk of data;
computer code for transmitting the initial data tuple for the chunk of data to the target node;
computer code for maintaining the initial data tuple at both the source node and the target node thereby forming a maintained source data tuple and a maintained target data tuple, wherein the maintained source data tuple includes a maintained source hash value and a maintained source sequence number corresponding to the initial sequence number, and wherein the maintained target data tuple includes a maintained target hash value and a maintained target sequence number corresponding to the initial sequence number;
computer code for receiving a request to determine whether data corresponding to the chunk at the source node is synchronized with the data corresponding to the chunk at the target node, the request corresponding to the maintained source data tuple and the maintained target data tuple;
computer code for determining whether the data at the source node is synchronized with the data at the target node by comparing a current source sequence number, separate from the maintained source sequence number, at the source node to the maintained source sequence number at the source node to indicate whether the data chunk has been modified at the source node, by comparing a current target sequence number, separate from the maintained target sequence number, at the target node to the maintained target sequence number at the target node to determine whether the data chunk has been modified at the target node, and by comparing the maintained source data tuple at the source node to the maintained target data tuple at the target node to determine if the data chunk is out-of-sync, wherein the current source sequence number is separate from the current target sequence number; and
computer code for performing resynchronization if the data at the source node is determined to not be synchronized with the data at the target node.