System and method for XOR chain
One embodiment is related to a method for creating a redundancy data chunk for data protection with a chain topology, comprising: transmitting a data chunk of a first frontend zone of a data storage system to a second frontend zone of the data storage system; creating a redundancy data chunk at the second frontend zone of the data storage system based on the data chunk of the first frontend zone and a data chunk of the second frontend zone; passing the redundancy data chunk onto one or more subsequent frontend zones of the data storage system from the second frontend zone, wherein at each subsequent frontend zone the redundancy data chunk is updated based on the received redundancy data chunk and a data chunk of the respective subsequent frontend zone, and wherein the redundancy data chunk is passed through each subsequent frontend zone exactly once; and at a last subsequent frontend zone of the data storage system, forwarding the updated redundancy data chunk to a backend zone of the data storage system for final storage.
1. A method for creating a redundancy data chunk for data protection with a chain topology, comprising:
transmitting a data chunk of a first frontend zone of a data storage system to a second frontend zone of the data storage system;
creating a redundancy data chunk at the second frontend zone of the data storage system based on the data chunk of the first frontend zone and a data chunk of the second frontend zone;
passing the redundancy data chunk onto one or more subsequent frontend zones of the data storage system from the second frontend zone, wherein at each subsequent frontend zone the redundancy data chunk is updated based on the received redundancy data chunk and a data chunk of the respective subsequent frontend zone, and wherein the redundancy data chunk is passed through each subsequent frontend zone exactly once; and
at a last subsequent frontend zone of the data storage system, forwarding the updated redundancy data chunk to a backend zone of the data storage system for final storage, wherein the backend zone is dedicated to storage of the redundancy data chunk generated based on an exclusive or (XOR) technique and does not host user data, and wherein the frontend and backend zones of the data storage system are geographically distributed.
2. The method of claim 1 , wherein the redundancy data chunk is an exclusive or (XOR) chunk.
3. The method of claim 2 , wherein the XOR chunk is updated at a particular frontend zone by performing a bitwise XOR operation on the received XOR chunk and a data chunk of the particular frontend zone.
4. The method of claim 1 , further comprising after the redundancy data chunk is stored in the backend zone, recovering a data chunk of a single failed frontend zone based on the redundancy data chunk and data chunks of healthy frontend zones.
5. The method of claim 4 , wherein during data recovery, the redundancy data chunk is passed through each healthy frontend zone exactly once.
6. The method of claim 5 , wherein at each healthy frontend zone, the respective frontend zone withdraws its data chunk from the redundancy data chunk.
7. A non-transitory machine-readable medium having instructions stored therein which, when executed by a processor, cause the processor to perform testing operations, the operations comprising:
transmitting a data chunk of a first frontend zone of a data storage system to a second frontend zone of the data storage system;
creating a redundancy data chunk at the second frontend zone of the data storage system based on the data chunk of the first frontend zone and a data chunk of the second frontend zone;
passing the redundancy data chunk onto one or more subsequent frontend zones of the data storage system from the second frontend zone, wherein at each subsequent frontend zone the redundancy data chunk is updated based on the received redundancy data chunk and a data chunk of the respective subsequent frontend zone, and wherein the redundancy data chunk is passed through each subsequent frontend zone exactly once; and
at a last subsequent frontend zone of the data storage system, forwarding the updated redundancy data chunk to a backend zone of the data storage system for final storage, wherein the backend zone is dedicated to storage of the redundancy data chunk generated based on an exclusive or (XOR) technique and does not host user data, and wherein the frontend and backend zones of the data storage system are geographically distributed.
8. The non-transitory machine-readable medium of claim 7 , wherein the redundancy data chunk is an exclusive or (XOR) chunk.
9. The non-transitory machine-readable medium of claim 8 , wherein the XOR chunk is updated at a particular frontend zone by performing a bitwise XOR operation on the received XOR chunk and a data chunk of the particular frontend zone.
10. The non-transitory machine-readable medium of claim 7 , wherein the operations further comprise after the redundancy data chunk is stored in the backend zone, recovering a data chunk of a single failed frontend zone based on the redundancy data chunk and data chunks of healthy frontend zones.
11. The non-transitory machine-readable medium of claim 10 , wherein during data recovery, the redundancy data chunk is passed through each healthy frontend zone exactly once.
12. The non-transitory machine-readable medium of claim 11 , wherein at each healthy frontend zone, the respective frontend zone withdraws its data chunk from the redundancy data chunk.
13. A data processing system, comprising:
a processor; and
a memory coupled to the processor storing instructions which, when executed by the processor, cause the processor to perform testing operations, the operations including
transmitting a data chunk of a first frontend zone of a data storage system to a second frontend zone of the data storage system;
creating a redundancy data chunk at the second frontend zone of the data storage system based on the data chunk of the first frontend zone and a data chunk of the second frontend zone;
passing the redundancy data chunk onto one or more subsequent frontend zones of the data storage system from the second frontend zone, wherein at each subsequent frontend zone the redundancy data chunk is updated based on the received redundancy data chunk and a data chunk of the respective subsequent frontend zone, and wherein the redundancy data chunk is passed through each subsequent frontend zone exactly once; and
at a last subsequent frontend zone of the data storage system, forwarding the updated redundancy data chunk to a backend zone of the data storage system for final storage, wherein the backend zone is dedicated to storage of the redundancy data chunk generated based on an exclusive or (XOR) technique and does not host user data, and wherein the frontend and backend zones of the data storage system are geographically distributed.
14. The data processing system of claim 13 , wherein the redundancy data chunk is an exclusive or (XOR) chunk.
15. The data processing system of claim 14 , wherein the XOR chunk is updated at a particular frontend zone by performing a bitwise XOR operation on the received XOR chunk and a data chunk of the particular frontend zone.
16. The data processing system of claim 13 , wherein the operations further comprise after the redundancy data chunk is stored in the backend zone, recovering a data chunk of a single failed frontend zone based on the redundancy data chunk and data chunks of healthy frontend zones.
17. The data processing system of claim 16 , wherein during data recovery, the redundancy data chunk is passed through each healthy frontend zone exactly once.
18. The data processing system of claim 17 , wherein at each healthy frontend zone, the respective frontend zone withdraws its data chunk from the redundancy data chunk.