Reducing the amount of data stored in a sequence of data blocks by combining deduplication and compression
The described technology is generally directed towards reducing the amount of data stored in a sequence of data blocks by combining deduplication and compression. According to an embodiment, a system can comprise a memory that can store computer executable components, and a processor that can execute the components stored in the memory. The components can comprise a data block identifier that can identify, for a sequence of data blocks, a first data block that corresponds to a first data, resulting in a first identified data block, and a deduplication component that can identify a second data block that corresponds to the first data, resulting in a second identified data block, wherein the deduplication component can replace the second identified data block with a key value corresponding to the first identified data block. Further, a compression component can compress the first identified data block, resulting in a compressed data block.
1. A method, comprising:
identifying, by a device comprising a processor, for a sequence of data elements, a first data element in the sequence of data elements that corresponds to first data, resulting in a first identified data element, and a second data element in the sequence of data elements that corresponds to the first data, resulting in a second identified data element; and
replacing, by the device, the second identified data element with a key value corresponding to the first identified data element.
2. The method of claim 1 , further comprising compressing, by the device, the first identified data element, resulting in a compressed data element.
3. The method of claim 2 , further comprising replacing, by the device, the first identified data element with the compressed data element.
4. The method of claim 2 , further comprising tagging, by the device, the first identified data element to be compressed, and wherein the compressing the first identified data element comprises compressing the first identified data element based on the tagging of the first identified data element.
5. The method of claim 4 , wherein the first identified data element comprises a header portion that comprises metadata, wherein the tagging of the first identified data element comprises tagging the first identified data element by updating the metadata, resulting in updated metadata, and wherein the compressing the first identified data element comprises compressing the first identified data element based on the updated metadata.
6. The method of claim 2 , wherein the sequence of data elements are comprised in a search tree data structure, wherein the identifying the first data element in the sequence of data elements comprises identifying the first data element based on a first search of the search tree data structure, wherein the identifying the second data element comprises identifying the second data element based on a second search of the search tree data structure, and wherein the method further comprises storing, by the device, the compressed data element in the search tree data structure.
7. The method of claim 6 , further comprising, based on the first search of the search tree data structure, tagging, by the device, the first identified data element to be compressed, and wherein the compressing the first identified data element comprises compressing the first identified data element based on the tagging of the first identified data element.
8. The method of claim 6 , wherein the search tree data structure comprises a binary search tree data structure.
9. The method of claim 2 , further comprising:
decompressing, by the device, the compressed data element resulting in a decompressed data element;
replacing, by the device, the compressed data element with the decompressed data element;
identifying, by the device for the sequence of data elements, the key value; and
based on the key value, replacing, by the device, the key value with a copy of the decompressed data element.
10. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
receiving a sequence of elements from a device, wherein respective ones of the sequence of elements comprise blocks of data;
decompressing a compressed block of data of a first element of the sequence of elements, resulting in a first block of data comprised in the first element; and
storing a copy of the first block of data as a second block of data comprised in a second element, wherein the storing the first block of data is based on a key value referencing the first element.
11. The system of claim 10 , wherein the respective ones of the sequence of elements further comprise metadata that describes contents of the blocks of data comprised in the respective ones of the sequence of elements.
12. The system of claim 11 , wherein the metadata comprised in the first element of the sequence of elements comprises a first indicator that the first element comprises compressed data, and wherein the decompressing the compressed block of data is based on the first indicator.
13. The system of claim 12 , wherein the metadata comprised in the first element comprises a value corresponding to a length of the compressed block of data.
14. The system of claim 11 , wherein the metadata comprised in the first element of the sequence of elements comprises a second indicator that the second element comprises the key value referencing the first element, and wherein the storing the copy of the first block of data is based on the second indicator.
15. The system of claim 10 , wherein the operations further comprise copying data corresponding to the sequence of elements to a track of a storage device.
16. The system of claim 10 , wherein the device is comprised in a data protection system and the sequence of elements corresponds to data replicated to the device based on the data protection system.
17. The system of claim 10 , wherein the sequence of elements comprises metadata that describes aspects of the sequence of elements.
18. A non-transitory machine-readable medium comprising executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising:
identifying, for a sequence of data elements, a first data element in the sequence of data elements that corresponds to first data, resulting in a first identified data element;
identifying a second data element in the sequence of data elements that corresponds to the first data, resulting in a second identified data element; and
replacing the second identified data element with a key value corresponding to the first identified data element.
19. The non-transitory machine-readable medium of claim 18 , wherein the operations further comprise tagging the first identified data element to be compressed, and wherein the operations further comprise compressing the first identified data element is based on the tagging of the first identified data element.
20. The non-transitory machine-readable medium of claim 19 , wherein the first identified data element comprises a header portion that comprises metadata, wherein the tagging of the first identified data element comprises updating the metadata, resulting in updated metadata, and wherein the compressing the first identified data element is based on the updated metadata.