Data convolution for geographically diverse storage
View Patent ↗Data convolution for geographically diverse storage is disclosed. Data chunks stored in storage devices of different zones of a zone storage system can be convolved to conserve memory. The zone storage system can be a geographically diverse storage system. A convolved chunk can be stored at a zone that does not contribute to the data represented in the convolved chunk. In an aspect, a copy of a first chunk from a first zone can be created at a third zone, which can facilitate convolving the data of the copy with data from a second zone as the data from the second chunk of the second zone is received at the third zone, e.g., without having to create a local copy of the second chunk, which can be termed ‘on-arrival convolving’ or other similar terms. The copy of the first chunk can then be deleted. Copies of the convolved chunk can be created.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
receiving, at a third zone storage component of the system, at least a first portion of first data comprised in a first data chunk from a first zone storage component of the system, wherein the first data chunk is immutable;
generating, at the third zone storage component, a replica of the first portion of the first data, resulting in a fourth data chunk stored on one or more hard disks at the third zone storage component, wherein the fourth data chunk is immutable;
receiving, at the third zone storage component, at least a second portion of second data comprised in a second data chunk from a second zone storage component of the system, wherein the second data chunk is immutable; and
generating a third data chunk at the third zone storage component based on the fourth data chunk and the at least the second portion of the second data, wherein the generating excludes creating a local copy of the second data chunk at the third zone storage component, and wherein the third data chunk is distinct from the fourth data chunk.
2. The system of claim 1 , wherein the system is a distributed storage system, wherein the first zone storage component corresponds to a first zone of the distributed storage system, wherein the third storage component corresponds to a third zone of the distributed storage system, and wherein the first zone is a different zone than the third zone.
3. The system of claim 2 , wherein the second zone storage component corresponds to a second zone of the distributed storage system, and wherein the second zone is a different zone than the third zone.
4. The system of claim 2 , wherein the third data chunk is replicated as a fourth data chunk.
5. The system of claim 4 , wherein the fourth data chunk is stored at a fourth zone storage component corresponding to the third zone of the distributed storage system, and wherein the third zone storage component is a different storage component than the fourth zone storage component.
6. The system of claim 1 , wherein the third data chunk results from an XOR combination of at least the first portion of the first data and at least the second portion of the second data.
7. The system of claim 2 , wherein the operations further comprise, in response to the receiving at least the first portion of the first data, generating a fourth data chunk at the third zone storage component, the fourth data chunk being a replication of the first data chunk.
8. The system of claim 7 , wherein the operations further comprise, in response to the generating the third data chunk, deleting the fourth data chunk.
9. The system of claim 8 , wherein the third data chunk is replicated as a fifth data chunk stored at a fourth zone storage component corresponding to the third zone of the distributed storage system, and wherein the third zone storage component is a different storage component than the fourth zone storage component.
10. The system of claim 1 , wherein the local copy of the second portion of the second data is a second local copy, wherein the operations further comprise creating a first local copy of the first data chunk at the third zone storage component in response to receiving the first portion of the first data, and wherein the generating the third data chunk comprises convolving the first local copy of the first data chunk with the second portion of the second data in response to arrival of the second portion of the second data at the third zone storage component.
11. The system of claim 1 , wherein the first zone storage component, the second zone storage component, and the third zone storage component are geographically distinct.
12. A method, comprising:
in response to receiving, by a system comprising a processor and a memory, at least a first portion of first data comprised in a first data chunk from a first zone storage component corresponding to a first zone of a distributed storage system, generating a first replica of the first data chunk based on at least the first portion of the first data, resulting in a fourth data chunk stored on a third zone storage component corresponding to a third zone of the distributed storage system, wherein the third zone is a different zone than the first zone, and wherein the first data chunk and the fourth data chunk cannot be modified;
receiving, by the system, at least a second portion of second data comprised in a second data chunk from a second zone storage component corresponding to a second zone of the distributed storage system, wherein the second zone is a different zone than the first zone, wherein the second zone is a different zone than the third zone, and wherein the second data chunk cannot be modified; and
generating, by the system, a third data chunk based on the fourth data chunk and at least the second portion of the second data, wherein the generating does not result in a second replica of the second data chunk, wherein the third data chunk is stored on the third zone storage component, and wherein the third data chunk is distinct from the fourth data chunk.
13. The method of claim 12 , wherein the generating the third data chunk comprises convolving the fourth data chunk and at least the second portion of the second data.
14. The method of claim 13 , wherein the convolving is via an XOR function.
15. The method of claim 12 , further comprising, in response to the generating the third data chunk, releasing, by the system, storage space of the third zone storage component corresponding to storing the fourth data chunk.
16. A non-transitory machine-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
generating a fourth data chunk based on at least a first portion of first data comprised in a first data chunk from a first zone storage component corresponding to a first zone of a distributed storage system, wherein the first data chunk and the fourth data chunk are unmodifiable, wherein the fourth data chunk is a copy of the first data chunk, wherein the fourth data chunk is stored on a third zone storage component corresponding to a third zone of the distributed storage system, and wherein the third zone is a different zone than the first zone; and
generating a third data chunk based on the fourth data chunk and at least a second portion of second data comprised in a second data chunk from a second zone storage component corresponding to a second zone of the distributed storage system without creating a local copy of the second data chunk on the third zone storage component, wherein the second data chunk is unmodifiable, wherein the second zone is a different zone than the first zone, wherein the second zone is a different zone than the third zone, wherein the third data chunk is stored on the third zone storage component, and wherein the third data chunk is distinct from the fourth data chunk.
17. The non-transitory machine-readable storage medium of claim 16 , wherein the generating the third data chunk comprises convolving the fourth data chunk and at least the second portion of the second data.
18. The non-transitory machine-readable storage medium of claim 17 , wherein the convolving is performed by applying an XOR logic to the fourth data chunk and at least the second portion of the second data.
19. The non-transitory machine-readable storage medium of claim 16 , wherein the operations further comprise, in response to the generating the third data chunk, deleting the fourth data chunk.
20. The non-transitory machine-readable storage medium of claim 16 , wherein the operations further comprise generating a copy of the third data chunk, resulting in a fifth data chunk, wherein the fifth data chunk is stored on a fourth zone storage component corresponding to the third zone of a distributed storage system, and wherein the fourth zone storage component is a different zone storage component than the third zone storage component.