High-density compression method and computing system
Certain implementations of the disclosed technology may include methods and computing systems for performing high-density data compression, particularly on numerical data that demonstrates various patterns, and patterns of patters. According to an example implementation, a method is provided. The method may include extracting a data sample from a data set, compressing the data sample using a first compression filter configuration, and calculating a compression ratio associated with the first compression filter configuration. The method may also include compressing the data sample using a second compression filter configuration and calculating a compression ratio associated with the second compression filter configuration. A particular compression filter configuration to utilize in compressing the entire data set may be selected based on a comparison of the compression ratio associated with the first compression filter configuration and a compression ratio associated with the second compression filter configuration.
1. A method comprising:
extracting, by a processor, a data sample from a set of uncompressed data;
compressing, by the processor, the data sample using a first compression filter configuration comprising a first plurality of different compression filters arranged in a first sequence;
calculating, by the processor, a first compression ratio associated with the first compression filter configuration;
compressing, by the processor, the data sample using a second compression filter configuration comprising a second plurality of different compression filters arranged in a second sequence;
calculating, by the processor, a second compression ratio associated with the second compression filter configuration;
comparing, by the processor, the first compression ratio with the second compression ratio;
selecting, by the processor, the first compression filter configuration or the second compression filter configuration based on the comparison of the first and second compression ratios to provide a selected compression filter configuration; and
compressing, by the processor, the set of uncompressed data using the selected compression filter configuration to provide a compressed data set.
2. The method of claim 1 , wherein the first plurality of different compression filters is the same as the second plurality of different compression filters and wherein the first sequence is different than the second sequence.
3. The method of claim 1 , wherein at least one of the first plurality of different compression filters is selected from a group consisting of:
dictionary run-length encoding;
dictionary run-length encoding of run-length encoding dictionaries; and
week-of encoding.
4. The method of claim 1 , wherein at least one of the second plurality of different compression filters is selected from a group consisting of:
dictionary run-length encoding;
dictionary run-length encoding of run-length encoding dictionaries; and
week-of encoding.
5. The method of claim 1 , further comprising:
reducing, by the processor, a size of the compressed data set by performing at least one of:
dictionary run-length encoding index omission;
dictionary run-length encoding byte shrinking;
data byte shrinking; and
range encoding.
6. A computing system comprising:
memory comprising executable instructions; and
a processor operatively connected to the memory, the processor configured to execute the executable instructions in order to effectuate a method comprising:
extracting a data sample from a set of uncompressed data;
compressing the data sample using a first compression filter configuration comprising a first plurality of different compression filters arranged in a first sequence;
calculating a first compression ratio associated with the first compression filter configuration;
compressing the data sample using a second compression filter configuration comprising a second plurality of different compression filters arranged in a second sequence;
calculating a second compression ratio associated with the second compression filter configuration;
comparing the first compression ratio with the second compression ratio;
selecting the first compression filter configuration or the second compression filter configuration based on the comparison of the first and second compression ratios to provide a selected compression filter configuration; and
compressing the set of uncompressed data using the selected compression filter configuration to provide a compressed data set.
7. The computing system of claim 6 , wherein the first plurality of different compression filters is the same as the second plurality of different compression filters and wherein the first sequence is different than the second sequence.
8. The computing system of claim 6 , wherein at least one of the first plurality of different compression filters is selected from a group consisting of:
dictionary run-length encoding;
dictionary run-length encoding of run-length encoding dictionaries; and
week-of encoding.
9. The computing system of claim 6 , wherein at least one of the second plurality of different compression filters is selected from a group consisting of:
dictionary run-length encoding;
dictionary run-length encoding of run-length encoding dictionaries; and
week-of encoding.
10. The computing system of claim 6 , wherein the processor is configured to execute the executable instructions in order to effectuate the method further comprising:
reducing, by the processor, a size of the compressed data set by performing at least one of:
dictionary run-length encoding index omission;
dictionary run-length encoding byte shrinking;
data byte shrinking; and
range encoding.
11. A non-transitory computer-readable medium comprising executable instructions that when executed by a processor cause the processor to effectuate a method comprising:
extracting a data sample from a set of uncompressed data;
compressing the data sample using a first compression filter configuration comprising a first plurality of different compression filters arranged in a first sequence;
calculating a first compression ratio associated with the first compression filter configuration;
compressing the data sample using a second compression filter configuration comprising a second plurality of different compression filters arranged in a second sequence;
calculating a second compression ratio associated with the second compression filter configuration;
comparing the first compression ratio with the second compression ratio;
selecting the first compression filter configuration or the second compression filter configuration based on the comparison of the first and second compression ratios to provide a selected compression filter configuration; and
compressing the set of uncompressed data using the selected compression filter configuration to provide a compressed data set.
12. The non-transitory computer-readable medium of claim 11 , wherein the first plurality of different compression filters is the same as the second plurality of different compression filters and wherein the first sequence is different than the second sequence.
13. The non-transitory computer-readable medium of claim 11 , wherein at least one of the first plurality of different compression filters is selected from a group consisting of:
dictionary run-length encoding;
dictionary run-length encoding of run-length encoding dictionaries; and
week-of encoding.
14. The non-transitory computer-readable medium of claim 11 , wherein at least one of the second plurality of different compression filters is selected from a group consisting of:
dictionary run-length encoding;
dictionary run-length encoding of run-length encoding dictionaries; and
week-of encoding.
15. The non-transitory computer-readable medium of claim 11 , wherein executing the executable instructions causes the processor to effectuate the method further comprising:
reducing, by the processor, a size of the compressed data set by performing at least one of:
dictionary run-length encoding index omission;
dictionary run-length encoding byte shrinking;
data byte shrinking; and
range encoding.