Content-aware compression of floating-point time-series data using multiple prediction functions and estimated bit-saving thresholds
Methods and apparatus are provided for content-aware compression of data using multiple prediction functions. An exemplary method comprises obtaining a floating point number; applying a default prediction algorithm and at least one other distinct prediction algorithm to the obtained floating point number to generate a plurality of predictions; determining a residual for each prediction based on a difference between the predictions and the floating point number; determining a bit savings estimate for each prediction based on a difference between an exponent of each prediction and an exponent of the residual for each prediction; selecting the default prediction algorithm or one other distinct prediction algorithm for encoding the floating point number based on the determined bit savings estimate for each prediction; and encoding the floating point number by encoding the determined residual associated with the selected prediction algorithm and/or the determined bit savings estimate associated with the selected prediction algorithm.
1. A method for compressing at least one floating point number, comprising the steps of:
obtaining said at least one floating point number comprising a sign, an exponent at a given base and a significand;
applying a default prediction algorithm and at least one other distinct prediction algorithm to said obtained at least one floating point number to generate a plurality of predictions;
determining a residual for each of said plurality of predictions based on a difference between said plurality of predictions and said at least one floating point number;
determining a bit savings estimate for each of said plurality of predictions based on a difference between an exponent of each said plurality of predictions and an exponent of said residual for each of said plurality of predictions;
selecting, using at least one processing device, a given one of said default prediction algorithm and said at least one other distinct prediction algorithm for encoding said at least one floating point number based on said determined bit savings estimate for each of said plurality of predictions; and
encoding, using the at least one processing device, said at least one floating point number by encoding one or more of said determined residual associated with said selected prediction algorithm and said determined bit savings estimate associated with said selected prediction algorithm.
2. The method of claim 1 , further comprising the step of encoding an index of said at least one other distinct prediction algorithm when said at least one other distinct prediction algorithm is said selected prediction algorithm.
3. The method of claim 2 , wherein said index comprises a disambiguation index indicating one selected prediction algorithm from a potential subset of prediction algorithms.
4. The method of claim 3 , wherein said potential subset is generated by discarding the predictions that, when added to said determined residual, correspond to a floating point number for which there would be a better prediction resulting in another residual that can be represented with fewer bits.
5. The method of claim 1 , wherein a given one of said at least one other distinct prediction algorithm is selected based on whether said determined bit savings estimate for said given one of said at least one other distinct prediction algorithm is greater than a first threshold.
6. The method of claim 1 , wherein a given one of said at least one other distinct prediction algorithm is selected based on whether a relative bit savings estimate for said given one of said at least one other distinct prediction algorithm with respect to said default prediction algorithm is greater than a second threshold.
7. The method of claim 1 , wherein the exponent of said determined residual associated with said selected prediction algorithm is replaced with said determined bit savings estimate associated with said selected prediction algorithm.
8. The method of claim 1 , further comprising the step of decompressing said encoded at least one floating point number by decoding said determined residual and said determined bit savings estimate associated with said selected prediction algorithm to determine whether said selected prediction algorithm was said default prediction algorithm or said at least one other distinct prediction algorithm.
9. The method of claim 8 , further comprising the step of reading an index of said at least one other distinct prediction algorithm when said selected prediction algorithm was said at least one other distinct prediction algorithm based on whether said decoded determined bit savings estimate for a given one of said at least one other distinct prediction algorithm is greater than a first threshold and on whether a relative bit savings estimate for said given one of said at least one other distinct prediction algorithm with respect to said default prediction algorithm is greater than a second threshold.
10. The method of claim 8 , further comprising the step of restoring the exponent of said determined residual by adding said decoded bit savings and the exponent of said selected prediction.
11. The method of claim 1 , further comprising the step of determining a set of one or more prediction algorithms out of a larger set of prediction algorithms for a specific data set including said at least one floating point number based on an analysis of said specific data set.
12. The method of claim 11 , wherein one or more of said (i) default prediction algorithms; (ii) said at least one other distinct prediction algorithm; (iii) a first threshold value for said determined bit savings estimate for said at least one other distinct prediction algorithm; and (iv) a second threshold value for a relative bit savings estimate for said at least one other distinct prediction algorithm with respect to said default prediction algorithm are selected for a segment of said specific data set.
13. A computer program product for compressing at least one floating point number, comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:
obtaining said at least one floating point number comprising a sign, an exponent at a given base and a significand;
applying a default prediction algorithm and at least one other distinct prediction algorithm to said obtained at least one floating point number to generate a plurality of predictions;
determining a residual for each of said plurality of predictions based on a difference between said plurality of predictions and said at least one floating point number;
determining a bit savings estimate for each of said plurality of predictions based on a difference between an exponent of each said plurality of predictions and an exponent of said residual for each of said plurality of predictions;
selecting a given one of said default prediction algorithm and said at least one other distinct prediction algorithm for encoding said at least one floating point number based on said determined bit savings estimate for each of said plurality of predictions; and
encoding said at least one floating point number by encoding one or more of said determined residual associated with said selected prediction algorithm and said determined bit savings estimate associated with said selected prediction algorithm.
14. A system for compressing at least one floating point number, comprising:
a memory; and
at least one processing device, coupled to the memory, operative to implement the following steps:
obtaining said at least one floating point number comprising a sign, an exponent at a given base and a significand;
applying a default prediction algorithm and at least one other distinct prediction algorithm to said obtained at least one floating point number to generate a plurality of predictions;
determining a residual for each of said plurality of predictions based on a difference between said plurality of predictions and said at least one floating point number;
determining a bit savings estimate for each of said plurality of predictions based on a difference between an exponent of each said plurality of predictions and an exponent of said residual for each of said plurality of predictions;
selecting a given one of said default prediction algorithm and said at least one other distinct prediction algorithm for encoding said at least one floating point number based on said determined bit savings estimate for each of said plurality of predictions; and
encoding said at least one floating point number by encoding one or more of said determined residual associated with said selected prediction algorithm and said determined bit savings estimate associated with said selected prediction algorithm.
15. The system of claim 14 , further comprising the step of encoding an index of said at least one other distinct prediction algorithm when said at least one other distinct prediction algorithm is said selected prediction algorithm.
16. The system of claim 14 , wherein a given one of said at least one other distinct prediction algorithm is selected based on whether said determined bit savings estimate for said given one of said at least one other distinct prediction algorithm is greater than a first threshold.
17. The system of claim 14 , wherein a given one of said at least one other distinct prediction algorithm is selected based on whether a relative bit savings estimate for said given one of said at least one other distinct prediction algorithm with respect to said default prediction algorithm is greater than a second threshold.
18. The system of claim 14 , further comprising the step of decompressing said encoded at least one floating point number by decoding said determined residual and said determined bit savings estimate associated with said selected prediction algorithm to determine whether said selected prediction algorithm was said default prediction algorithm or said at least one other distinct prediction algorithm.
19. The system of claim 18 , further comprising one or more steps of reading an index of said at least one other distinct prediction algorithm when said selected prediction algorithm was said at least one other distinct prediction algorithm based on whether said decoded determined bit savings estimate for a given one of said at least one other distinct prediction algorithm is greater than a first threshold and on whether a relative bit savings estimate for said given one of said at least one other distinct prediction algorithm with respect to said default prediction algorithm is greater than a second threshold; and restoring the exponent of said determined residual by adding said decoded bit savings and the exponent of said selected prediction.
20. The system of claim 14 , wherein one or more of said (i) default prediction algorithms; (ii) said at least one other distinct prediction algorithm; (iii) a first threshold value for said determined bit savings estimate for said at least one other distinct prediction algorithm; and (iv) a second threshold value for a relative bit savings estimate for said at least one other distinct prediction algorithm with respect to said default prediction algorithm are selected for a segment of a specific data set including said at least one floating point number.