Ability to maintain RPO in clustered environment with failed nodes/disks
A method for maintaining a replica of a primary storage on a secondary storage, when the primary storage includes data stored in a plurality of nodes. When at least one node or disk has failed, the method takes partial snapshot that excludes the failed node or disk. A diff file is created by the differences between the partial snapshot and a snapshot taken immediately prior to the failure. The diff is applied to the replica. When the failed node or disk comes back online, the method takes a snapshot prior to allowing any write operation. A diff file of the new snapshot is generated by comparing the new snapshot to the snapshot taken immediately prior to the failure. The new diff is applied to the replica.
1. A system comprising:
a primary storage having data stored in a plurality of nodes;
a secondary storage storing a replica of the primary storage;
one or more processors; and
a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:
periodically update the replica and when it is determined that at least one of the plurality of nodes is offline, performing the steps:
taking a partial snapshot of the primary storage excluding offline node;
forming a partial diff file by comparing the snapshot to immediately preceding snapshot;
sending the partial diff file to the secondary storage.
2. The system of claim 1 , wherein the plurality of instructions, when executed, further cause the one or more processors to apply the partial diff file to the replica.
3. The system of claim 2 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
store identification of the offline node.
4. The system of claim 3 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
when receiving an indication that the offline node is operational, performing the steps:
identify directory node in the primary storage;
notify directory node that offline node is operational.
5. The system of claim 4 , wherein the plurality of instructions, when executed, further cause the one or more processors to:
when receiving notification that offline node is operational performing the steps:
taking node snapshot of the offline node;
generating a node diff file by comparing the node snapshot to the partial snapshot;
sending the node diff file to the secondary storage.
6. The system of claim 5 , comprising:
wherein the plurality of instructions, when executed, further cause the one or more processors to apply the node diff to the replica.
7. In a system having a primary storage having data stored in a plurality of nodes and a secondary storage storing a replica of the primary storage, a computer implemented method comprising:
periodically update the replica and when it is determined that at least one of the plurality of nodes is offline, performing the steps:
taking a partial snapshot of the primary storage excluding offline node;
forming a partial diff file by comparing the snapshot to immediately preceding snapshot;
sending the partial diff file to the secondary storage.
8. The method of claim 7 , further comprising applying the partial diff file to the replica.
9. The method of claim 8 , further comprising storing identification of the offline node.
10. The method of claim 9 , further comprising when receiving an indication that the offline node is operational, performing the steps:
identifying directory node in the primary storage;
notifying directory node that offline node is operational.
11. The method of claim 10 , further comprising
when receiving notification that offline node is operational performing the steps:
taking node snapshot of the offline node;
generating a node diff file by comparing the node snapshot to the partial snapshot;
sending the node diff file to the secondary storage.
12. The method of claim 11 , further comprising applying the node diff to the replica.
13. A computer program product comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:
periodically update a replica of a primary storage in a secondary storage, the primary storage having a plurality of nodes;
when it is determined that at least one of the plurality of nodes is offline, performing the steps:
taking a partial snapshot of the primary storage excluding offline node;
forming a partial diff file by comparing the snapshot to immediately preceding snapshot;
sending the partial diff file to the secondary storage.
14. The computer program product of claim 13 , wherein the program code includes further instructions to apply the partial diff file to the replica.
15. The computer program product of claim 14 , wherein the program code includes further instructions to store identification of the offline node.
16. The computer program product of claim 15 , wherein the program code includes further instructions to: when receiving an indication that the offline node is operational, performing the steps:
identify directory node in the primary storage;
notify directory node that offline node is operational.
17. The computer program product of claim 16 , wherein the program code includes further instructions to: when receiving notification that offline node is operational performing the steps:
taking node snapshot of the offline node;
generating a node diff file by comparing the node snapshot to the partial snapshot;
sending the node diff file to the secondary storage.
18. The computer program product of claim 17 , the program code including instructions to apply the node diff to the replica.