LZSS with multiple dictionaries and windows
A method and apparatus for compressing data is described. An input string to be compressed is received. The input string is encoded with compressor using a compression algorithm using several sizes of dictionaries and windows. The compressor processes the input string with a selected size of the dictionary and window yielding the most compression of the input string among the different sizes of dictionaries and windows.
1. A computer-implemented method comprising:
receiving an input string to be compressed;
encoding the input string with a compression algorithm with a plurality of dictionaries and windows of different sizes using a compressor; and
processing the input string with a selected size of the dictionary and window yielding the most compression of the input string among the different sizes of the plurality of dictionaries and windows.
2. The computer-implemented method of claim 1 wherein encoding further comprises:
generating a single dictionary of a maximum size;
processing the input string multiple times in parallel;
pausing when a predetermined amount of input has been processed; and
outputting a coded block that corresponds to the best compression result.
3. The computer-implemented method of claim 2 wherein each block is prefixed with a header that specifies the size of the coded block, and the dictionary and window sizes in use.
4. The computer-implemented method of claim 3 wherein a current block is encoded without using a previous dictionary.
5. The computer-implemented method of claim 1 wherein encoding further comprising:
processing a single compression pass on a block of input data using the maximum dictionary and window sizes; and
running a post-processing pass over a compression output to see if at least one of the entire dictionary and window size was used.
6. The computer-implemented method of claim 5 further comprising:
re-processing the coded output with at least one of a smaller dictionary and/or window size when the entire dictionary or window size was not used.
7. The computer-implemented method of claim 1 wherein the compression algorithm replaces a current string being encoded with an offset to a matching previously seen string and its length.
8. A non-transitory computer-accessible storage medium including data that, when accessed by a system, cause the system to perform a method comprising:
receiving an input string to be compressed;
encoding the input string with a compression algorithm with a plurality of dictionaries and windows of different sizes; and
processing the input string with a selected size of the dictionary and window yielding the most compression of the input string among the different sizes of the plurality of dictionaries and windows.
9. The non-transitory computer-accessible storage medium of claim 8 wherein encoding further comprises:
generating a single dictionary of a maximum size;
processing the input string multiple times in parallel;
pausing when a predetermined amount of input has been processed; and
outputting a coded block that corresponds to the best compression result.
10. The non-transitory computer-accessible storage medium of claim 9 wherein each block is prefixed with a header that specifies the size of the coded block, and the dictionary and window sizes in use.
11. The non-transitory computer-accessible storage medium of claim 10 wherein a current block is encoded without using a previous dictionary.
12. The non-transitory computer-accessible storage medium of claim 8 wherein encoding further comprising:
processing a single compression pass on a block of input data using the maximum dictionary and window sizes; and
running a post-processing pass over a compression output to see if at least one of the entire dictionary and window size was actually used.
13. The non-transitory computer-accessible storage medium of claim 12 further comprising:
re-processing the coded output with at least one of a smaller dictionary and window size when the entire dictionary or window size was not used.
14. The non-transitory computer-accessible storage medium of claim 8 wherein the compression algorithm replaces a current string being encoded with an offset to a matching previously seen string and its length.
15. An apparatus comprising:
a compression encoder configured to receive an input string to be compressed, to encode the input string with a compression algorithm with a plurality of dictionaries and windows of different sizes, to process the input string with a selected size of the dictionary and window yielding the most compression of the input string among the different sizes of the plurality of dictionaries and windows; and
a dictionary and window size generator coupled to the compression encoder.
16. The apparatus of claim 15 wherein the dictionary and window size generator is configured to generate a single dictionary of a maximum size, wherein the compression encoder is configured to process the input string multiple times in parallel, to pause when a predetermined amount of input has been processed, and to output a coded block that corresponds to the best compression result.
17. The apparatus of claim 16 wherein each block is prefixed with a header that specifies the size of the coded block, and the dictionary and window sizes in use.
18. The apparatus of claim 17 wherein a current block is encoded without using a previous dictionary.
19. The apparatus of claim 15 wherein the compression encoder is configured to process a single compression pass on a block of input data using the maximum dictionary and window sizes, and to run a post-processing pass over a compression output to see if at least one of the entire dictionary and window size was actually used.
20. The apparatus of claim 19 wherein the compression encoder is configured to re-process the coded output with at least one of a smaller dictionary and window size when the entire dictionary or window size was not used.