Data compression for multidimensional time series data
Described herein are computer-implemented methods for compressing sparse multidimensional ordered series data. In particular, these methods and apparatuses for performing them (including software) may be particularly well suited to efficiently compressing spectrographic data.
1. A computer-implemented method for compressing sparse multidimensional ordered series data, the method comprising:
dividing a multidimensional ordered series data into a plurality of local regions, wherein the data in each local region comprise one or more indexed data sets, each indexed data set comprising an index (n) within a given local region of the sparse multidimensional ordered series data and one or more variables that are indexed by the index (n);
identifying a predictor that calculates each of the one or more variables as a function of the index (n), a current local region data, and at least one previous local region data;
adjusting the current local region data by subtracting a scaled predicted correlated data based on the at least one previous local region data when a level of correlation exceeding a threshold exists between the current local region data and the previous local region data; and
encoding the adjusted current local region data, including an optimum scale factor, into a compressed stream.
2. The method of claim 1 , wherein the multidimensional ordered series data is spectrographic data.
3. The method of claim 1 , wherein dividing comprises dividing the data into a plurality of overlapping local regions.
4. The method of claim 1 , wherein dividing comprises dividing the data into a plurality of non-overlapping local regions.
5. The method of claim 1 , further comprising computing a correlation level between the current local region data and the previous local region data.
6. The method of claim 5 , wherein the correlation level is computed from a series of similar peaks that exist between the current local region data and the at least one previous local regions data.
7. The method of claim 6 , wherein the series of similar peaks is considered similar if a majority of peaks from the series of similar peaks have one or more of: approximately a same mass-to-charge ratio, approximately a same charge state as determined from spacing between subsequent peaks, and similar peak intensity abundance distributions that match an avergine model.
8. The method of claim 1 , wherein an encoder encodes an identifier identifying the at least one previous local region data.
9. The method of claim 1 , further comprising processing the plurality of local regions in an order, wherein the steps of identifying the predictor, adjusting the current local region data and encoding the adjusted current local region data are repeated for each local region in the order.
10. The method of claim 9 , wherein the order is a scan order or raster-scan order.
11. The method of claim 9 , wherein the order is selected from an order having a highest correlation level.
12. A computer-implemented method for compressing sparse multidimensional ordered series data, the method comprising:
dividing a multidimensional ordered series data comprising histopathological imaging data into a plurality of local regions, wherein the data in each local region comprise one or more indexed data sets, each indexed data set comprising a spatial index (n) within a given local region of the histopathological imaging data and one or more variables that are indexed by the spatial index (n); and
processing the plurality of local regions in an order by:
identifying a predictor that calculates each of the one or more variables as a function of the spatial index (n), a current local region data, and at least one previous local region data;
adjusting the current local region data by subtracting a scaled predicted correlated data based on the at least one previous local region data when a level of correlation exceeding a threshold exists between the current local region data and the previous local region data; and
encoding the adjusted current local region data, including an optimum scale factor, into a compressed stream.
13. A system for compressing sparse multidimensional ordered series data, the system comprising a non-transitory computer-readable medium with instructions stored thereon, that when executed by a processor, cause the processor to:
divide a multidimensional ordered series data into a plurality of local regions, wherein the data in each local region comprise one or more indexed data sets, each indexed data set comprising an index (n) within a given local region of the sparse multidimensional ordered series data and one or more variables that are indexed by the index (n);
identify a predictor that calculates each of the one or more variables as a function of the index (n), a current local region data, and at least one previous local region data;
adjust the current local region data by subtracting a scaled predicted correlated data based on the at least one previous local region data when a level of correlation exceeding a threshold exists between the current local region data and the previous local region data; and
encode the adjusted current local region data, including an optimum scale factor, into a compressed stream.
14. The system of claim 13 , wherein the multidimensional ordered series data is spectrographic data.
15. The system of claim 13 , wherein the processor is configured to encode an indicator of the previous local region data into the compressed stream.
16. The system of claim 13 , wherein dividing comprises dividing the data into a plurality of overlapping local regions.
17. The system of claim 13 , wherein dividing comprises dividing the data into a plurality of non-overlapping local regions.
18. The system of claim 13 , wherein the processor is further configured to compute a correlation level between the current local region data and the previous local region data.
19. The system of claim 18 , wherein the correlation level is computed from a series of similar peaks that exist between the current local region data and the at least one previous local regions data.
20. The system of claim 19 , wherein the series of similar peaks is considered similar if a majority of peaks from the series of similar peaks have one or more of: approximately a same mass-to-charge ratio, approximately a same charge state as determined from spacing between subsequent peaks, and similar peak intensity abundance distributions that match an avergine model.