Internet of things data compression system and method

1. A method for lossless data compression, comprising:

receiving a data block of one-dimensional data from sensor devices or gauges by a processor;

performing, by the processor, a sparse transform of extraction on the data block, wherein the sparse transform is defined as Y=AX, wherein A is defined as an nxM sparse transfer matrix, n is a number of elements in a subset, and M is the number of elements in an original set;

selecting, by the processor, a transform matrix for the data block;

modeling, by the processor, the selected transform matrix for the data block;

selecting, by the processor, a transform coefficient model for the data block;

modeling, by the processor, the selected transform coefficient model for the data block; and

compressing, by the processor, the data in the data block using the selected transform matrix and the selected transform coefficient model.

2. The method of claim 1 , wherein performing the sparse transform extraction is performed by finding non-zero components of a sparse transform matrix C from Y=CX, where X is a vector of possible values of a data type in the data block and Y is a vector of the data block.

3. The method of claim 1 , wherein selecting a transform matrix comprises selecting a direct transform model for the data block.

4. The method of claim 3 , wherein selecting a direct transform model comprises:

determining a size of the data block,

defining a first threshold size percentage that can be applied to the data block,

calculating the threshold size of the data block,

determining unique values in the data block, and

determining that the number of unique values in the data block is greater than the first threshold size of the data block.

5. The method of claim 1 , wherein selecting a transform matrix comprises selecting a dominant value first decomposed transform model.

6. The method of claim 5 , wherein selecting a dominant value first decomposed transform model comprises:

determining a size of the data block,

defining a second threshold size percentage that can be applied to the data block,

calculating the threshold size of the data block,

determining unique values in the data block, and

determining that the number of unique values in the data block is less than a second threshold size of the data block.

7. The method of claim 1 , wherein selecting a transform matrix comprises selecting a natural ordered decomposed transform model.

8. The method of claim 7 , wherein selecting a natural ordered decomposed transform model comprises:

defining a first threshold size percentage that can be applied to the data block,

calculating the threshold size of the data block,

defining a second threshold size percentage that can be applied to the data block,

calculating the threshold size of the data block,

determining unique values in the data block,

determining that the number of unique values in the data block is neither greater than the first threshold size of the data block nor less than a second threshold size of the data block;

calculating linear prediction residuals for the number of unique values in the data block; and

determining a power of the linear prediction residuals is less than a power of the unique values of the data block.

9. The method of claim 1 , wherein selecting a transform matrix comprises selecting an ascending ordered decomposed transform model.

10. The method of claim 9 , wherein selecting an ascending ordered decomposed transform model comprises determining the number of unique values in the data block is neither greater than a first threshold size of a size of the data block nor less than a second threshold size of a size of the data block;

calculating linear prediction residuals for the number of unique values in the data block; and

determining a power of the linear prediction residuals is greater than a power of the unique values of the data block.

11. The method of claim 1 , wherein selecting a transform coefficient model comprises selecting a prediction model.

12. The method of claim 1 , wherein selecting a transform coefficient model comprises selecting a granularity constrain model.

13. The method of claim 1 , wherein selecting a transform coefficient model comprises selecting a variation constrain model.

14. The method of claim 1 , wherein selecting a transform coefficient model comprises selecting a dynamic limitation model.

15. A lossless data compression system, comprising:

a gateway comprising a processor that can receive a stream of one-dimensional data from sensor devices or gauges,

wherein the processor is configured to compress data using a multichannel data compression encoder by:

receiving a data block;

performing a sparse transform extraction on the data block, wherein the sparse transform is defined as Y=AX, wherein A is defined as an nxM sparse transfer matrix, n is a number of elements in a subset, and M is the number of elements in an original set;

selecting a transform matrix for the data block;

modeling the selected transform matrix for the data block;

selecting a transform coefficient model for the data block;

modeling the selected transform coefficient model for the data block; and

compressing the data in the data block using the selected transform matrix and the selected transform coefficient model.

16. The system of claim 15 , wherein the processor performing the sparse transform extraction comprises finding non-zero components of a sparse transform matrix C from Y=CX, where X is a vector of possible values of a data type in the data block and Y is a vector of the data block.

17. The system of claim 15 , wherein the processor selecting a transform matrix comprises selecting a direct transform model for the data block.

18. The system of claim 17 , wherein the processor selecting a direct transform model comprises:

determining a size of the data block,

defining a first threshold size percentage that can be applied to the data block,

calculating the threshold size of the data block,

determining unique values in the data block, and

determining that the number of unique values in the data block is greater than the first threshold size of the data block.

19. The system of claim 15 , wherein the processor selecting a transform matrix comprises selecting a dominant value first decomposed transform model.

20. The system of claim 19 , wherein the processor selecting a dominant value first decomposed transform model comprises:

determining a size of the data block,

defining a second threshold size percentage that can be applied to the data block,

calculating the threshold size of the data block,

determining unique values in the data block, and

determining that the number of unique values in the data block is less than a second threshold size of the data block.

21. The system of claim 15 , wherein the processor selecting a transform matrix comprises selecting a natural ordered decomposed transform model.

22. The system of claim 21 , wherein the processor selecting a natural ordered decomposed transform model comprises:

defining a first threshold size percentage that can be applied to the data block,

calculating the threshold size of the data block,

defining a second threshold size percentage that can be applied to the data block,

calculating the threshold size of the data block,

determining unique values in the data block,

determining that the number of unique values in the data block is neither greater than the first threshold size of the data block nor less than a second threshold size of the data block;

calculating linear prediction residuals for the number of unique values in the data block; and

determining a power of the linear prediction residuals is less than a power of the unique values of the data block.

23. The system of claim 15 , wherein the processor selecting a transform matrix comprises selecting an ascending ordered decomposed transform model.

24. The system of claim 23 , wherein the processor selecting an ascending ordered decomposed transform model comprises determining the number of unique values in the data block is neither greater than a first threshold size of a size of the data block nor less than a second threshold size of a size of the data block;

calculating linear prediction residuals for the number of unique values in the data block; and

determining a power of the linear prediction residuals is greater than a power of the unique values of the data block.

25. The system of claim 15 , wherein the processor selecting a transform coefficient model comprises selecting a prediction model.

26. The system of claim 15 , wherein the processor selecting a transform coefficient model comprises selecting a granularity constrain model.

27. The system of claim 15 , wherein the processor selecting a transform coefficient model comprises selecting a variation constrain model.

28. The system of claim 15 , wherein the processor selecting a transform coefficient model comprises selecting a dynamic limitation model.