System and method for hash-based entropy calculation
A method, computer program product, and computing system for receiving a candidate data portion; calculating a distance-preserving hash for the candidate data portion; and performing an entropy analysis on the distance-preserving hash to generate a hash entropy for the candidate data portion.
1. A computer-implemented method, executed on a computing device, comprising:
receiving a candidate data portion;
calculating a distance-preserving hash for the candidate data portion;
performing an entropy analysis on the distance-preserving hash to generate a hash entropy for the candidate data portion; and
comparing the hash entropy for the candidate data portion to a predefined hash entropy threshold;
performing one of a data compression operation and a deduplication operation based upon, at least in part, comparing the has entropy for the candidate data portion to the predefined hash entropy threshold, wherein the predefined hash entropy threshold is configured such that:
a data compression operation will yield a more efficient result than the deduplication operation when the candidate data portion has a hash entropy that adheres to the hash entropy threshold; and
a deduplication operation will yield a more efficient result than the data compression operation when the candidate data portion has a hash entropy above the hash entropy threshold.
2. The computer-implemented method of claim 1 further comprising:
if the hash entropy for the candidate data portion adheres to the predefined hash entropy threshold, compressing the candidate data portion.
3. The computer-implemented method of claim 1 further comprising:
performing a comparison analysis on the one or more potential target data portions, with respect to the candidate data portion, to identify an acceptable target data portion.
4. The computer-implemented method of claim 3 further comprising:
performing a deduplication operation of the candidate data portion based upon the acceptable target data portion.
5. The computer-implemented method of claim 1 wherein the distance-preserving hash includes a cyclic redundancy check.
6. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:
receiving a candidate data portion;
calculating a distance-preserving hash for the candidate data portion;
performing an entropy analysis on the distance-preserving hash to generate a hash entropy for the candidate data portion; and
comparing the hash entropy for the candidate data portion to a predefined hash entropy threshold;
performing one of a data compression operation and a deduplication operation based upon, at least in part, comparing the has entropy for the candidate data portion to the predefined hash entropy threshold, wherein the predefined hash entropy threshold is configured such that:
a data compression operation will yield a more efficient result than the deduplication operation when the candidate data portion has a hash entropy that adheres to the hash entropy threshold; and
a deduplication operation will yield a more efficient result than the data compression operation when the candidate data portion has a hash entropy above the hash entropy threshold.
7. The computer program product of claim 6 further comprising:
if the hash entropy for the candidate data portion adheres to the predefined hash entropy threshold, compressing the candidate data portion.
8. The computer program product of claim 6 further comprising:
performing a comparison analysis on the one or more potential target data portions, with respect to the candidate data portion, to identify an acceptable target data portion.
9. The computer program product of claim 8 further comprising:
performing a deduplication operation of the candidate data portion based upon the acceptable target data portion.
10. The computer program product of claim 6 wherein the distance-preserving hash includes a cyclic redundancy check.
11. A computing system including a hardware processor and memory configured to perform operations comprising:
receiving a candidate data portion;
calculating a distance-preserving hash for the candidate data portion;
performing an entropy analysis on the distance-preserving hash to generate a hash entropy for the candidate data portion; and
comparing the hash entropy for the candidate data portion to a predefined hash entropy threshold;
performing one of a data compression operation and a deduplication operation based upon, at least in part, comparing the has entropy for the candidate data portion to the predefined hash entropy threshold, wherein the predefined hash entropy threshold is configured such that:
a data compression operation will yield a more efficient result than the deduplication operation when the candidate data portion has a hash entropy that adheres to the hash entropy threshold; and
a deduplication operation will yield a more efficient result than the data compression operation when the candidate data portion has a hash entropy above the hash entropy threshold.
12. The computing system of claim 11 further comprising:
if the hash entropy for the candidate data portion adheres to the predefined hash entropy threshold, compressing the candidate data portion.
13. The computing system of claim 12 further comprising:
performing a comparison analysis on the one or more potential target data portions, with respect to the candidate data portion, to identify an acceptable target data portion.
14. The computing system of claim 13 further comprising:
performing a deduplication operation of the candidate data portion based upon the acceptable target data portion.
15. The computing system of claim 11 wherein the distance-preserving hash includes a cyclic redundancy check.