Systems and methods for lossless compression of tabular numeric data
Systems and methods for lossless compression of tabular numeric data are provided. The system can include one or more data compression servers executing data compression system code to compress the tabular numeric data, a storage database to store the compressed tabular numeric data, and one or more data decompression servers to decompress the tabular numeric data for use. The one or more data compression servers, the storage database, and the one or more data decompression servers can communicate via a communication network. The system can receive the uncompressed tabular numeric data from one or more data generation systems, processes the uncompressed tabular numeric data with the data compression system code, and generate a compressed table of numeric information, which can be stored in the database, or later decompressed by the one or more data decompression servers.
1. A system for lossless compression of tabular numeric data, comprising:
a memory; and
a processor in communication with the memory, the processor:
receiving a table of uncompressed numeric information;
generating a table of integers based on the table of uncompressed numeric information;
rewriting each row in the table of integers based on a difference between values in a first row and values in a preceding row;
converting an absolute value of each number in the table of integers into one or more strings;
concatenating the strings for each number;
writing a byte in a new byte array for each pair of characters in the concatenated string; and
exporting the new byte array as a compressed data file.
2. The system of claim 1 , wherein the processor stores a sign of each number in the table of integers.
3. The system of claim 1 , wherein the processor calculates the absolute value of each number in table of integers.
4. The system of claim 1 , wherein the processor concatenates the strings for each number by concatenating a base-14 string with separators based on the sign of each number.
5. The system of claim 4 , wherein the processor reserves the new byte array with a length that is half of the concatenated base-14 string.
6. The system of claim 5 , wherein the processor writes the byte in the new byte array for each pair of characters in the concatenated base-14 string.
7. A method for lossless compression of tabular numeric data, comprising the steps of:
receiving at a processor a table of uncompressed numeric information;
generating by the processor a table of integers based on the table of uncompressed numeric information;
rewriting by the processor each row in the table of integers based on a difference between values in a first row and values in a preceding row;
converting by the processor an absolute value of each number in the table of integers into one or more strings;
concatenating by the processor the strings for each number;
writing by the processor a byte in a new byte array for each pair of characters in the concatenated string; and
exporting by the processor the new byte array as a compressed data file.
8. The method of claim 7 , further comprising storing by the processor a sign of each number in the table of integers.
9. The method of claim 7 , further comprising calculating by the processor the absolute value of each number in table of integers.
10. The method of claim 7 , further comprising concatenating by the processor the strings for each number by concatenating a base-14 string with separators based on the sign of each number.
11. The method of claim 10 , further comprising reserving by the processor the new byte array with a length that is half of the concatenated base-14 string.
12. The method of claim 11 , further comprising writing by the processor the byte in the new byte array for each pair of characters in the concatenated base-14 string.
13. A non-transitory, computer-readable medium having computer-readable instructions stored thereon which, when executed by a processor, cause the processor to perform the steps of:
receiving at the processor a table of uncompressed numeric information;
generating by the processor a table of integers based on the table of uncompressed numeric information;
rewriting by the processor each row in the table of integers based on a difference between values in a first row and values in a preceding row;
converting by the processor an absolute value of each number in the table of integers into one or more strings;
concatenating by the processor the strings for each number;
writing by the processor a byte in a new byte array for each pair of characters in the concatenated string; and
exporting by the processor the new byte array as a compressed data file.
14. The computer-readable medium of claim 13 , further comprising storing by the processor a sign of each number in the table of integers.
15. The computer-readable medium of claim 13 , further comprising calculating by the processor the absolute value of each number in table of integers.
16. The computer-readable medium of claim 13 , further comprising concatenating by the processor the strings for each number by concatenating a base-14 string with separators based on the sign of each number.
17. The computer-readable medium of claim 16 , further comprising reserving by the processor the new byte array with a length that is half of the concatenated base-14 string.
18. The computer-readable medium of claim 17 , further comprising writing by the processor the byte in the new byte array for each pair of characters in the concatenated base-14 string.