Content-aware compression of data using multiple parallel prediction functions
View Patent ↗Multiple parallel prediction functions are employed for content-aware data compression. An exemplary method comprises obtaining a floating point number comprising a sign, an exponent at a given base and a significand having a length; applying a plurality of distinct prediction algorithms to the floating point number to generate a corresponding plurality of predictions; selecting a given one of the plurality of distinct prediction algorithms for the floating point number by evaluating a compression metric applied to the plurality of predictions; and encoding the floating point number by encoding the exponent and the length as a single code using a residual generated by the selected prediction algorithm. A disambiguation index optionally identifies the selected prediction algorithm among a set of prediction algorithms that potentially generated the selected prediction. A set of one or more predictors out of a larger set of predictors can be determined for a specific data set based on a performance-based ranking.
1. A method for compressing at least one floating point number, comprising the steps of:
obtaining said at least one floating point number represented using one or more bits to indicate a sign of said at least one floating point number and one or more additional bits to indicate an exponent at a given base and a significand of said at least one floating point number, wherein said significand has a length equal to a number of bits between a most significant bit of said significand and a least significant bit of said significand having a predefined binary value;
applying, using at least one processing device, a plurality of distinct prediction algorithms to said at least one floating point number to generate a corresponding plurality of predictions;
selecting, using the at least one processing device, a given one of said plurality of distinct prediction algorithms for said at least one floating point number by evaluating a compression metric applied to said plurality of predictions; and
encoding, using the at least one processing device, said at least one floating point number by encoding said exponent and said length as a single code using a residual generated by said selected prediction algorithm.
2. The method of claim 1 , wherein said compression metric is related to selecting said given one of said plurality of predictions that generates a residual of smallest magnitude and a largest bit saving in a number of bits to be encoded.
3. The method of claim 1 , further comprising the step of storing an indication of said selected prediction algorithm.
4. The method of claim 3 , wherein said indication comprises a disambiguation index identifying said selected prediction algorithm among a set of potential prediction algorithms that potentially generated said selected prediction.
5. The method of claim 4 , wherein said set of potential prediction algorithms is generated by discarding one or more predictions that, when added to said residual, correspond to a floating point number for which a better prediction algorithm results in another residual with fewer bits.
6. The method of claim 3 , further comprising the step of decompressing said compressed given floating point number by restoring discarded bits based on said encoded length and said stored indication.
7. The method of claim 1 , further comprising the step of determining a set of one or more predictors out of a larger set of predictors for a specific data set including said at least one floating point number based on a performance-based ranking.
8. The method of claim 7 , wherein said performance-based ranking selects the predictors in the set of predictors based on an extent that the selected predictors complement one another such that fewer bits result overall when residuals of a set of said floating point numbers are considered.
9. The method of claim 7 , wherein said set of one or more predictors is selected for a segment of said data set including said at least one floating point number, wherein said data set is segmented based on one or more of a local variance, a local average, a local measure of smoothness and a local auto-correlation.
10. The method of claim 9 , further comprising the step of storing a disambiguation index indicating one selected prediction algorithm from among the set of predictors selected for said segment.
11. The method of claim 10 , further comprising the step of decompressing said compressed given floating point number by restoring discarded bits based on said encoded length and said disambiguation index, considering the set of predictors selected for said segment to which said compressed given floating point number belongs.
12. A computer program product for compressing at least one floating point number, comprising a tangible machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:
obtaining said at least one floating point number represented using one or more bits to indicate a sign of said at least one floating point number and one or more additional bits to indicate an exponent at a given base and a significand of said at least one floating point number, wherein said significand has a length equal to a number of bits between a most significant bit of said significand and a least significant bit of said significand having a predefined binary value;
applying, using at least one processing device, a plurality of distinct prediction algorithms to said at least one floating point number to generate a corresponding plurality of predictions;
selecting, using the at least one processing device, a given one of said plurality of distinct prediction algorithms for said at least one floating point number by evaluating a compression metric applied to said plurality of predictions; and
encoding, using the at least one processing device, said at least one floating point number by encoding said exponent and said length as a single code using a residual generated by said selected prediction algorithm.
13. The computer program product of claim 12 , wherein said compression metric is related to selecting said given one of said plurality of predictions that generates a residual of smallest magnitude and a largest bit saving in a number of bits to be encoded.
14. The computer program product of claim 12 , further comprising the step of storing an indication of said selected prediction algorithm.
15. The computer program product of claim 12 , further comprising the step of determining a set of one or more predictors out of a larger set of predictors for a specific data set including said at least one floating point number based on a performance-based ranking.
16. A system for compressing at least one floating point number, comprising:
a memory; and
at least one processing device, coupled to the memory, operative to implement the following steps:
obtaining said at least one floating point number represented using one or more bits to indicate a sign of said at least one floating point number and one or more additional bits to indicate an exponent at a given base and a significand of said at least one floating point number, wherein said significand has a length equal to a number of bits between a most significant bit of said significand and a least significant bit of said significand having a predefined binary value;
applying, using the at least one processing device, a plurality of distinct prediction algorithms to said at least one floating point number to generate a corresponding plurality of predictions;
selecting, using the at least one processing device, a given one of said plurality of distinct prediction algorithms for said at least one floating point number by evaluating a compression metric applied to said plurality of predictions; and
encoding, using the at least one processing device, said at least one floating point number by encoding said exponent and said length as a single code using a residual generated by said selected prediction algorithm.
17. The system of claim 16 , wherein said compression metric is related to selecting said given one of said plurality of predictions that generates a residual of smallest magnitude and a largest bit saving in a number of bits to be encoded.
18. The system of claim 16 , further comprising the step of storing an indication of said selected prediction algorithm.
19. The system of claim 18 , wherein said indication comprises a disambiguation index identifying said selected prediction algorithm among a set of potential prediction algorithms that potentially generated said selected prediction.
20. The system of claim 18 , further comprising the step of decompressing said compressed given floating point number by restoring discarded bits based on said encoded length and said stored indication.
21. The system of claim 16 , further comprising the step of determining a set of one or more predictors out of a larger set of predictors for a specific data set including said at least one floating point number based on a performance-based ranking.
22. The system of claim 21 , wherein said set of one or more predictors is selected for a segment of said data set including said at least one floating point number, wherein said data set is segmented based on one or more of a local variance, a local average, a local measure of smoothness and a local auto-correlation.