Immediate replication for dedicated data blocks
The systems and methods disclosed herein relate to immediate replication for protected dedicated chunk. In one embodiment, a method comprises generating an encoded portion of data of a dedicated chunk in a node device of a first group of node devices in a first zone of a distributed storage system. The method further comprises, in immediate response to the generating, transferring the encoded portion of data of the dedicated chunk from the node device of the first group of node devices in the first zone of the distributed storage system to a second zone of the distributed storage system. The method further comprises dividing, by the system, the encoded portion of data in the second zone across second node devices.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
generating an encoded portion of data of a dedicated chunk in a node device of a first group of node devices in a first zone of a distributed storage system, wherein the encoded portion of the data comprises a group of data fragments and a group of coding fragments, and wherein the group of data fragments and the group of coding fragments are immediately allocated to volatile memory;
in immediate response to the generating, while the encoded portion of the data is available via the volatile memory, transferring the encoded portion of the data of the dedicated chunk from the node device of the first group of node devices in the first zone of the distributed storage system to a second zone of the distributed storage system after the encoded portion of the data has been protected in non-volatile memory and before the encoded portion of the data is discarded from the volatile memory; and
dividing the encoded portion of the data in the second zone across a second group of node devices of the second zone.
2. The system of claim 1 , wherein the operations further comprise:
generating the encoded portion of the data comprising executing an erasure encoding task on a portion of the data.
3. The system of claim 1 , wherein the operations further comprise:
storing the encoded portion of the data of the dedicated chunk in the node device of the first group of node devices.
4. The system of claim 1 , wherein the operations further comprise:
detecting the dedicated chunk from one or more chunks in which the encoded portion of the data for a single object is stored.
5. The system of claim 4 , wherein detecting the dedicated chunk comprises looking up an identification for the single object.
6. The system of claim 1 , wherein the operations further comprise:
determining whether the dedicated chunk is immutably sealed, prior to the transferring of the encoded portion of the data of a dedicated chunk from the node device of the first group of node devices in the first zone to the second zone.
7. The system of claim 6 , wherein the system tracks an amount of written data on the dedicated chunk prior to the dedicated chunk being immutably sealed.
8. A method, comprising:
generating, by a system comprising a processor, an encoded portion of data of a dedicated chunk in a node device of a first group of node devices in a first zone of a distributed storage system, wherein the encoded portion of the data comprises a group of data fragments and a group of coding fragments, and wherein the group of data fragments and the group of coding fragments are immediately designated to volatile memory;
in immediate response to the generating, while the encoded portion of the data is accessible via the volatile memory, transferring, by the system, the encoded portion of the data of the dedicated chunk from the node device of the first group of node devices in the first zone of the distributed storage system to a second zone of the distributed storage system after the encoded portion of the data has been protected in non-volatile memory and before the encoded portion of the data is discarded from the volatile memory; and
dividing, by the system, the encoded portion of the data in the second zone across second node devices.
9. The method of claim 8 , further comprising executing, by the system, an erasure encoding task on the portion of the data to generate the encoded portion of the data.
10. The method of claim 8 , further comprising storing, by the system, the encoded portion of the data of the dedicated chunk in the node device of the first node devices.
11. The method of claim 8 , further comprising detecting, by the system, the dedicated chunk from at least one chunk in which the encoded portion of the data for a single object is stored.
12. The method of claim 11 , wherein the detecting the dedicated chunk comprises looking up an identification for the single object.
13. The method of claim 8 , further comprising determining, by the system, whether the dedicated chunk is sealed according to a threshold immutability prior to the transferring of the encoded portion of the data of a dedicated chunk from the node device of the first node devices in the first zone to the second zone.
14. The method of claim 13 , further comprising tracking an amount of written data on the dedicated chunk prior to the dedicated chunk being immutably sealed.
15. A non-transitory computer-readable storage medium comprising instructions that, in response to execution, cause a device comprising a processor to perform operations, comprising:
generating an encoded portion of data of a dedicated chunk in a node device of a first group of node devices in a first zone of a distributed storage system, wherein the encoded portion of the data comprises a group of data fragments and a group of coding fragments, and wherein the group of data fragments and the group of coding fragments are immediately stored in volatile memory;
in immediate response to the generating, while the encoded portion of the data is accessible via the volatile memory, transferring the encoded portion of the data of the dedicated chunk from the node device of the first group of node devices in the first zone of the distributed storage system to a second zone of the distributed storage system after the encoded portion of the data has been protected in non-volatile memory and before the encoded portion of the data is discarded from the volatile memory; and
dividing the encoded portion of the data in the second zone across the second plurality of nodes.
16. The computer-readable storage medium of claim 15 , wherein the operations further comprise:
executing an erasure encoding task on a portion of data to generate the encoded portion of the data.
17. The computer-readable storage medium of claim 15 , wherein the operations further comprise:
storing the encoded portion of the data of the dedicated chunk in the node of the first plurality of nodes.
18. The computer-readable storage medium of claim 15 , wherein the operations further comprise:
detecting the dedicated chunk from chunks in which the encoded portion of the data for a single object is stored.
19. The computer-readable storage medium of claim 18 , wherein the detecting the dedicated chunk comprises looking up an identification for the single object.
20. The computer-readable storage medium of claim 15 , wherein the operations further comprise:
determining that the dedicated chunk is sealed, to confirm an immutability property of the dedicated chunk prior to the transferring of the encoded portion of the data of the dedicated chunk from the node of the first plurality of nodes in the first zone to the second zone.