Technologies for a high-ratio compression accelerator with heterogeneous history buffers
Technologies for high-ratio compression with heterogeneous history buffers include a computing device having an accelerator complex with a large history buffer and a small history buffer. The large history buffer has a larger size than the small history buffer. For example, the small history buffer may be 32 kilobytes and the large history buffer may be 64 kilobytes, 1 megabyte, or larger. The large history buffer is coupled to a large-buffer compare core that searches for matches in the large history buffer, finds a best match, and forwards the best match to a small-buffer compare core. The small-buffer compare core searches the small history buffer for matches, receives the match forwarded from the large-buffer compare core, and determines a best match from the matches in the small history buffer and the forwarded match. Other embodiments are described and claimed.
1. A computing device for data compression, the computing device comprising:
a first history buffer coupled to a first compare core, wherein the first history buffer has a first size; and
a second history buffer coupled to a second compare core, wherein the second history buffer has a second size that is less than the first size;
wherein the first compare core is to (i) search for a first match in the first history buffer, wherein the first match comprises a length and a backward distance, (ii) output the first match, and (iii) forward the first match to the second compare core, wherein to forward the first match comprises to reduce the length of the first match by an offset between a current input position of the first compare core and a current input position of the second compare core; and
wherein the second compare core is to (i) search for a second match in the second history buffer, wherein the second match comprises a length and a backward distance, (ii) select a best match from the first match and the second match, wherein the best match has a largest length of the first match and the second match, and (iii) output the best match.
2. The computing device of claim 1 , wherein:
the length and the backward distance of the first match identify a string in the first history buffer that matches a string of an uncompressed input data starting at the current input position of the first compare core; and
the length and the backward distance of the second match identify a string in the second history buffer that matches a string of the uncompressed input data starting at the current position of the second compare core.
3. The computing device of claim 1 , wherein the second size comprises a compression algorithm history window size.
4. The computing device of claim 3 , wherein the compression algorithm history window size comprises a DEFLATE algorithm history window size.
5. The computing device of claim 3 , wherein the second size comprises 32 kilobytes.
6. The computing device of claim 1 , wherein the first size comprises a compression algorithm history window size.
7. The computing device of claim 6 , wherein the compression algorithm history window size comprises a DEFLATE algorithm history window size.
8. The computing device of claim 6 , wherein the first size comprises 32 kilobytes.
9. The computing device of claim 1 , wherein the first size comprises 1 megabyte and the second size comprises 32 kilobytes.
10. The computing device of claim 1 , wherein:
the first compare core is further to (i) search for a plurality of matches in the first history buffer, wherein to search for the plurality of matches comprises to search for the first match, and (ii) select a best match from the plurality of matches, wherein the first match comprises the best match;
wherein to forward the first match to the second compare core comprises to forward the best match in response to selecting the best match.
11. The computing device of claim 1 , further comprising:
a plurality of compare cores, wherein each of the plurality of compare cores is coupled to a history buffer with a size that is less than the first size, and wherein the plurality of compare cores comprises the second compare core;
wherein the first compare core is further to forward the first match to the plurality of compare cores.
12. The computing device of claim 1 , further comprising a third compare core coupled to the first history buffer, wherein the third compare core is to (i) search for a third match in the first history buffer, and (ii) output the third match.
13. The computing device of claim 1 , further comprising a merge/coalesce logic to merge the first match output by the first compare core and the best match output by the second compare core to generate compressed output data.
14. A method for data compression, the method comprising:
searching, by a first compare core of a computing device, for a first match in a first history buffer, wherein the first history buffer has a first size, and wherein the first match comprises a length and a backward distance;
outputting, by the first compare core, the first match;
forwarding, by the first compare core, the first match to a second compare core of the computing device, wherein forwarding the first match comprises reducing the length of the first match by an offset between a current input position of the first compare core and a current input position of the second compare core;
searching, by the second compare core, for a second match in a second history buffer, wherein the second history buffer has a second size that is less than the first size, and wherein the second match comprises a length and a backward distance;
selecting, by the second compare core, a best match from the first match and the second match, wherein the best match has a largest length of the first match and the second match; and
outputting, by the second compare core, the best match.
15. The method of claim 14 , wherein:
the length and the backward distance of the first match identify a string in the first history buffer that matches a string of an uncompressed input data starting at the current input position of the first compare core; and
the length and the backward distance of the second match identify a string in the second history buffer that matches a string of the uncompressed input data starting at the current position of the second compare core.
16. The method of claim 14 , wherein the second size comprises a compression algorithm history window size.
17. The method of claim 14 , wherein the first size comprises a compression algorithm history window size.
18. The method of claim 14 , further comprising:
searching, by the first compare core, for a plurality of matches in the first history buffer, wherein searching for the plurality of matches comprises searching for the first match; and
selecting, by the first compare core, a best match from the plurality of matches, wherein the first match comprises the best match;
wherein forwarding the first match to the second compare core comprises forwarding the best match in response to selecting the best match.
19. The method of claim 14 , further comprising forwarding, by the first compare core, the first match to a plurality of compare cores of the computing device, wherein forwarding the first match to the plurality of compare cores comprises forwarding the first match to the second compare core, and wherein each of the plurality of compare cores searches a history buffer with a size that is less than the first size.
20. An accelerator complex for data compression, the accelerator complex comprising:
a first history buffer coupled to a first compare core, wherein the first history buffer has a first size; and
a second history buffer coupled to a second compare core, wherein the second history buffer has a second size that is less than the first size;
wherein the first compare core is to (i) search for a first match in the first history buffer, wherein the first match comprises a length and a backward distance, (ii) output the first match, and (iii) forward the first match to the second compare core, wherein to forward the first match comprises to reduce the length of the first match by an offset between a current input position of the first compare core and a current input position of the second compare core; and
wherein the second compare core is to (i) search for a second match in the second history buffer, wherein the second match comprises a length and a backward distance, (ii) select a best match from the first match and the second match, wherein the best match has a largest length of the first match and the second match, and (iii) output the best match.
21. The accelerator complex of claim 20 , further comprising an input buffer, wherein:
the length and the backward distance of the first match identify a string in the first history buffer that matches a string of in the input buffer starting at the current input position of the first compare core; and
the length and the backward distance of the second match identify a string in the second history buffer that matches a string in the input buffer starting at the current position of the second compare core.
22. The accelerator complex of claim 20 , wherein the second size comprises a compression algorithm history window size.
23. The accelerator complex of claim 20 , wherein the first size comprises a compression algorithm history window size.
24. The accelerator complex of claim 20 , wherein:
the first compare core is further to (i) search for a plurality of matches in the first history buffer, wherein to search for the plurality of matches comprises to search for the first match, and (ii) select a best match from the plurality of matches, wherein the first match comprises the best match;
wherein to forward the first match to the second compare core comprises to forward the best match in response to selecting the best match.
25. The accelerator complex of claim 20 , further comprising:
a plurality of compare cores, wherein each of the plurality of compare cores is coupled to a history buffer with a size that is less than the first size, and wherein the plurality of compare cores comprises the second compare core;
wherein the first compare core is further to forward the first match to the plurality of compare cores.