Handling deletes with distributed erasure coding
Inter-zone network traffic generated during deletion of a data chunk that has been replicated by employing geographically distributed (GEO) erasure coding is reduced. In one aspect, if a data chunk is to be deleted, partial coding chunks are generated by a source zone and provided to destination zones that store complete coding chunks for updating the complete coding chunks based on combining them with the received partial coding chunks. In another aspect, if a first data chunk is to be deleted and a second data chunk is to be replicated, partial coding chunks are generated by the source zone for each data chunk. Further, the partial coding chunks created for different data chunks can be combined to generate transforming chunks, which can then be transferred to the destination zones. The destination zones can then update the complete coding chunks based on combining them with the received transforming chunks.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
determining whether data chunks stored within a source zone of a geographically distributed storage system are to be modified;
in response to determining that data chunks stored within a source zone of a geographically distributed storage system are to be modified, determining coding information associated with a data chunk of the data chunks, wherein the determining the coding information is based on a coding matrix comprising a defined coefficient associated with the data chunk;
determining a destination zone of the geographically distributed storage system that stores a coding chunk associated with the data chunk, wherein the coding chunk has been determined based on an erasure coding operation associated with the data chunk; and
directing, to the destination zone, a portion of the coding information to facilitate a modification of the coding chunk, wherein the modification results in a removal of information related to the data chunk from the coding chunk.
2. The system of claim 1 , wherein the operations further comprise:
receiving a storage request from a storage client to delete data comprising the data chunk.
3. The system of claim 1 , wherein the directing comprises bi-directional communication between the source zone and the destination zone, and wherein a copy of the portion of the coding information is to be created remotely at the destination zone.
4. The system of claim 1 , wherein the modification comprises re-generation of the portion of the coding information for the data chunk of the data chunks to be modified.
5. The system of claim 4 , wherein the operations further comprise generating an updated coding chunk comprising the portion of the coding information.
6. The system of claim 4 , wherein the operations further comprise protecting a new data chunk, and wherein the re-generation further comprises re-generation of a portion of coding information for the new data chunk.
7. The system of claim 1 , wherein the operations further comprise adding a new data chunk to the source zone, wherein the adding comprises generating a new portion of coding information for the new data chunk.
8. The system of claim 1 , wherein the destination zone is a first destination zone, wherein remote zones comprise the first destination zone and a second destination zone of the system, and wherein the remote zones update the coding chunk independently.
9. A method, comprising:
in response to determining that a group of data chunks stored within a first zone of a geographically distributed storage system are to be modified, determining, by a system comprising a processor, coding information associated with a data chunk of the group of data chunks, wherein the determining the coding information is based on defined coding matrix data;
determining a second zone of the geographically distributed storage system that stores a coding chunk associated with the data chunk, wherein the coding chunk has been determined based on an erasure coding operation associated with the data chunk; and
directing, to the second zone, a portion of the coding information to facilitate a modification of the coding chunk, wherein the modification results in a removal of information related to the data chunk from the coding chunk.
10. The method of claim 9 , wherein the determining that the group of data chunks are to be modified comprises determining that the data chunk is to be deleted.
11. The method of claim 10 , wherein the defined coding matrix data comprises defined coefficient data and the determining the coding information comprises determining, based on the defined coefficient data, a group of partial coding chunks associated with the data chunk.
12. The method of claim 11 , wherein the directing comprises directing a partial coding chunk of the group of partial coding chunks to the second zone, and wherein the modification comprises a combination of the partial coding chunk with the coding chunk.
13. The method of claim 9 , wherein the data chunk is a first data chunk, and wherein the determining that the group of data chunks are to be modified comprises determining that the first data chunk is to be deleted and that a second data chunk of the group of data chunks is to be replicated.
14. The method of claim 13 , wherein the determining the coding information comprises:
determining a first partial coding chunk associated with the first data chunk;
determining a second partial coding chunk associated with the second data chunk; and
combining the first partial coding chunk and the second partial coding chunk to generate a transforming chunk.
15. The method of claim 14 , wherein the directing comprises directing the transforming chunk to the second zone.
16. The method of claim 15 , wherein the information is first information, wherein the modification results in addition of second information to the coding chunk, and wherein the second information is employable to recover the second data chunk during a failure.
17. A non-transitory computer-readable medium comprising instructions that, in response to execution, cause a server device comprising a processor to perform operations, comprising:
in response to determining that data chunks contained in a first zone of a storage system comprising a geographically distributed setup are to be modified,
determining, based on coding matrix data, coding information associated with a data chunk of the data chunks;
in response to the determining the coding information, determining a second zone of the storage system that contains a coding chunk associated with the data chunk;
locating the coding chunk in the second zone utilizing an erasure coding operation associated with the data chunk;
delivering, to the second zone, a portion of the coding information; and
in response to the delivering, modifying the coding chunk, wherein the modifying comprises removing information related to the data chunk from the coding chunk.
18. The non-transitory computer-readable medium of claim 17 , wherein
the determining that the data chunks are to be modified is based on instructions received from a client device that the data chunk is to be deleted,
the coding matrix data comprises defined integer coefficients,
the determining the coding information comprises determining, based on the coding matrix data having defined integer coefficients associated with the data chunk, partial coding chunks associated with the data chunk,
the delivering comprises delivering a partial coding chunk of the partial coding chunks to the second zone, and
the modifying further comprises subtracting the partial coding chunk from the coding chunk.
19. The non-transitory computer-readable medium of claim 17 , wherein
the data chunk is a first data chunk,
the determining that the data chunks are to be modified comprises determining based on instructions received that the first data chunk is to be deleted,
a second data chunk of the data chunks is to be replicated, and
the determining the coding information comprises:
generating a transforming chunk, comprising combining a first partial coding chunk and a second partial coding chunk, wherein
the first partial coding chunk is associated with the first data chunk, and
the second partial coding chunk is associated with the second data chunk.
20. The non-transitory computer-readable medium of claim 19 , wherein
the delivering comprises providing the transforming chunk to the second zone,
the modification results in addition of second information, different from the information to the coding chunk, and
the second data chunk is recoverable during a disaster by utilizing the second information.