System and method for sub-block deduplication with search for identical sectors inside a candidate block
A method, computer program product, and computer system for identifying duplicate sectors in a block of a plurality of blocks. The duplicate sectors in the block may be zeroed out. A data reduction operation may be performed on the block after the duplicate sectors are zeroed out.
1. A computer-implemented method comprising:
locally remapping one or more sectors in a first block of a plurality of blocks;
identifying duplicate sectors in the first block of the plurality of blocks including encoding target and candidate sector matches in a sector remap array wherein a sector remap array position implies a location of a matching candidate sector and a sector remap array content is an index of a matching sector in a partial dedupe target sector providing the matching data;
reflecting the duplicate sectors in metadata associated with the first block;
zeroing out the duplicate sectors in the first block; and
performing a data reduction operation on the first block after the duplicate sectors are zeroed out.
2. The computer-implemented method of claim 1 wherein the remapping is shared sector remapping and identifying the duplicate sectors includes the shared sector remapping.
3. The computer-implemented method of claim 2 wherein the shared sector remapping encodes duplicate sectors located in the first block.
4. The computer-implemented method of claim 2 wherein a negative index value indicates the duplicate sectors are located in the first block.
5. The computer-implemented method of claim 1 wherein performing the data reduction operation includes eliminating the zeroed out sectors using compression.
6. The computer-implemented method of claim 4 wherein signed bytes are used for a remapping array.
7. The computer-implemented method of claim 6 wherein the signed bytes indicate whether the duplicate sectors match.
8. A computer program product residing on a non-transitory computer readable storage medium having a plurality of instructions stored thereon which, when executed across one or more processors, causes at least a portion of the one or more processors to perform operations comprising:
locally remapping one or more sectors in a first block of a plurality of blocks;
identifying duplicate sectors in the first block of the plurality of blocks including encoding target and candidate sector matches in a sector remap array wherein a sector remap array position implies a location of a matching candidate sector and a sector remap array content is an index of a matching sector in a partial dedupe target sector providing the matching data;
reflecting the duplicate sectors in metadata associated with the first block;
zeroing out the duplicate sectors in the first block; and
performing a data reduction operation on the first block after the duplicate sectors are zeroed out.
9. The computer program product of claim 8 wherein the remapping is shared sector remapping and identifying the duplicate sectors includes the shared sector remapping.
10. The computer program product of claim 9 wherein the shared sector remapping encodes duplicate sectors located in the first block.
11. The computer program product of claim 9 wherein a negative index value indicates the duplicate sectors are located in the first block.
12. The computer program product of claim 8 wherein performing the data reduction operation includes eliminating the zeroed out sectors using compression.
13. The computer program product of claim 11 wherein signed bytes are used for a remapping array.
14. The computer program product of claim 13 wherein the signed bytes indicate whether the duplicate sectors match.
15. A computing system including one or more processors and one or more memories configured to perform operations comprising:
locally remapping one or more sectors in a first block of a plurality of blocks;
identifying duplicate sectors in the first block of the plurality of blocks including encoding target and candidate sector matches in a sector remap array wherein a sector remap array position implies a location of a matching candidate sector and a sector remap array content is an index of a matching sector in a partial dedupe target sector providing the matching data;
reflecting the duplicate sectors in metadata associated with the first block;
zeroing out the duplicate sectors in the first block; and
performing a data reduction operation on the first block after the duplicate sectors are zeroed out.
16. The computing system of claim 15 wherein the remapping is shared sector remapping, identifying the duplicate sectors includes the shared sector remapping, and wherein the shared sector remapping encodes duplicate sectors located in the first block.
17. The computing system of claim 16 wherein a negative index value indicates the duplicate sectors are located in the first block.
18. The computing system of claim 15 wherein performing the data reduction operation includes eliminating the zeroed out sectors using compression.
19. The computing system of claim 17 wherein signed bytes are used for a remapping array.
20. The computing system of claim 19 wherein the signed bytes indicate whether the duplicate sectors match.