Run-length base-delta encoding for high-speed compression

1. A method for high-speed data compression, the method comprising:

selecting a header based on an input stream, wherein the header includes a base, a scheme and a delta count, wherein a current value is used as a base;

determining whether there are any remaining values in an uncompressed input stream based on the selected header;

reading a first next value from the input stream;

determining whether the read first next value is representable with a current base scheme by verifying a resulting difference between the current value and the first next value can be represented by a number of bytes specified by the scheme;

calculating the delta count based on determining that the read first next value is representable with the current base scheme; and

writing the calculated delta count to the selected header.

2. The method of claim 1 , further comprising:

reading a second next value from an incremented delta count;

determining that the read second next value is not representable with the current base scheme;

writing the base, the scheme and a second next delta count based on the read second next value;

determining whether there are any remaining next values in the uncompressed input stream based on the written base, the written scheme and the written second next delta count; and

decompressing the current base scheme based on determining that there are no remaining next values in the uncompressed input stream.

3. The method of claim 2 , wherein decompressing the current base scheme further comprises:

initializing a number of bytes read to zero;

determining that the number of bytes read does not equal a compressed size;

reading the header based on the number of bytes read;

writing the base to a second output stream based on the read header;

determining whether the delta count is greater than zero based on the written base;

reading the delta size from a compressed input stream based on determining that the delta count is greater than zero; and

writing a base plus delta to an output based on the read delta size.

4. The method of claim 2 , wherein decompressing the current base scheme further comprises:

initializing a number of bytes read to zero;

determining that the number of bytes read does not equal a compressed size;

reading the header based on the number of bytes read;

writing the base to a second output stream based on the read header; and

determining whether the delta count is zero based on the written base.

5. The method of claim 2 , wherein decompressing the current base scheme further comprises:

initializing a number of bytes read to zero; and

determining whether the number of bytes read equals a compressed size.

6. The method of claim 1 , wherein a compress-transfer-decompress pipeline (CTDP) process is used to break a large data set transmitted between a computer memory and an accelerator device memory into a plurality of smaller data chunks.

7. The method of claim 6 , wherein the CTDP process further comprises;

dividing the data set into the plurality of smaller data chunks;

compressing the data chunks using a run-length base-delta compression algorithm;

transferring the compressed data chunks to a target;

decompressing the transferred data chunks using the run-length base-delta compression algorithm; and

reassembling the decompressed data chunks on a device to recover the data set.

8. The method of claim 7 , wherein the target includes a graphics processing unit (GPU) or a computing device with a separate memory.

9. A computer system for high-speed data compression, comprising:

one or more processors, one or more computer-readable memories, one or more non-transitory computer-readable tangible storage media, and program instructions stored on at least one of the one or more tangible storage media for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:

selecting a header based on an input stream, wherein the header includes a base, a scheme and a delta count, wherein a current value is used as a base;

determining whether there are any remaining values in an uncompressed input stream based on the selected header;

reading a first next value from the input stream;

determining whether the read first next value is representable with a current base scheme by verifying a resulting difference between the current value and the first next value can be represented by a number of bytes specified by the scheme;

calculating the delta count based on determining that the read first next value is representable with the current base scheme; and

writing the calculated delta count to the selected header.

10. The computer system of claim 9 , further comprising:

reading a second next value from an incremented delta count;

determining that the read second next value is not representable with the current base scheme;

writing the base, the scheme and a second next delta count based on the read second next value;

determining whether there are any remaining next values in the uncompressed input stream based on the written base, the written scheme and the written second next delta count; and

decompressing the current base scheme based on determining that there are no remaining next values in the uncompressed input stream.

11. The computer system of claim 10 , the base scheme decompression further comprising:

initializing a number of bytes read to zero;

determining that the number of bytes read does not equal a compressed size;

reading the header based on the number of bytes read;

writing the base to a second output stream based on the read header;

determining whether the delta count is greater than zero based on the written base;

reading the delta size from a compressed input stream based on determining that the delta count is greater than zero; and

writing a base plus delta to an output based on the read delta size.

12. The computer system of claim 10 , the base scheme decompression further comprising:

initializing a number of bytes read to zero;

determining that the number of bytes read does not equal a compressed size;

reading the header based on the number of bytes read;

writing the base to a second output stream based on the read header; and

determining whether the delta count is zero based on the written base.

13. The computer system of claim 10 , the base scheme decompression further comprising:

initializing a number of bytes read to zero; and

determining whether the initialized number of bytes read equals a compressed size.

14. The computer system of claim 9 , wherein a compress-transfer-decompress pipeline (CTDP) process is used to break a large data set transmitted between a computer memory and an accelerator device memory into a plurality of smaller data chunks.

15. The computer system of claim 14 , the CTDP process further comprising;

dividing the data set into the plurality of smaller data chunks;

compressing the data chunks using a run-length base-delta compression algorithm;

transferring the compressed data chunks to a target;

decompressing the transferred data chunks using the run-length base-delta compression algorithm; and

reassembling the decompressed data chunks on a device to recover the data set.

16. The computer system of claim 15 , wherein the target includes a graphics processing unit (GPU) or a computing device with a separate memory.

17. A computer program product for high-speed data compression, comprising:

one or more non-transitory computer-readable storage media and program instructions stored on at least one of the one or more tangible storage media, the program instructions executable by a processor to cause the processor to perform a method comprising:

selecting a header based on an input stream, wherein the header includes a base, a scheme and a delta count, wherein a current value is used as a base;

determining whether there are any remaining values in an uncompressed input stream based on the selected header;

reading a first next value from the input stream;

determining whether the read first next value is representable with a current base scheme by verifying a resulting difference between the current value and the first next value can be represented by a number of bytes specified by the scheme;

calculating the delta count based on determining that the read first next value is representable with the current base scheme; and

writing the calculated delta count to the selected header.

18. The computer program product of claim 17 , further comprising:

reading a second next value from an incremented delta count;

determining that the read second next value is not representable with the current base scheme;

writing the base, the scheme and a second next delta count based on the read second next value;

determining whether there are any remaining next values in the uncompressed input stream based on the written base, the written scheme and the written second next delta count; and

decompressing the current base scheme based on determining that there are no remaining next values in the uncompressed input stream.

19. The computer program product of claim 18 , the base scheme decompression further comprising:

initializing a number of bytes read to zero;

determining that the number of bytes read does not equal a compressed size;

reading the header based on the number of bytes read;

writing the base to a second output stream based on the read header;

determining whether the delta count is greater than zero based on the written base;

reading the delta size from a compressed input stream based on determining that the delta count is greater than zero; and

writing a base plus delta to an output based on the read delta size.

20. The computer program product of claim 18 , the base scheme decompression further comprising:

initializing a number of bytes read to zero;

determining that the number of bytes read does not equal a compressed size;

reading the header based on the number of bytes read;

writing the base to a second output stream based on the read header; and

determining whether the delta count is zero based on the written base.