Technologies for deterministic constant-time data compression
A compute device to generate deterministic compressed streams receives a current string to be matched to one or more prior instances of the current string, the current string being located within an input buffer and the one or more prior instances located within a history buffer. The compute device identifies a limited subset of index memory designated for storing pointers to the prior instances, identifying a reserved slop region in the index memory, and compares the current string to a prior instance, locating the at least one prior instance using at least one pointer to the at least one prior instance. The at least one pointer is stored within the limited subset of the index memory, and the compute device also prohibits use of any pointers stored in the reserved slop region of the index memory. Other embodiments are described and claimed.
1. A compute device to generate deterministic compressed streams, the compute device comprising:
a compare engine to:
receive a current string to be matched to one or more prior instances of the current string, the current string located within an input buffer and the one or more prior instances located within a history buffer;
identify a limited subset of an index memory of the compare engine designated for storing pointers to the one or more prior instances, wherein to identify the limited subset is to identify a reserved slop region in the index memory; and
compare the current string to at least one prior instance of the one or more prior instances, wherein to compare the current string is to locate the at least one prior instance using at least one pointer to the at least one prior instance, wherein the at least one pointer is stored within the limited subset of the index memory, and wherein to compare the current string is further to prohibit use of any pointers stored in the reserved slop region of the index memory.
2. The compute device of claim 1 , wherein the compare engine comprises a hardware accelerator.
3. The compute device of claim 1 , wherein to compare the current string comprises to traverse in an order that is a reverse of an initialization order and to stop the traverse in response to entering the reserved slop region.
4. The compute device of claim 1 , further comprising a table updater to index the input buffer at positions from the current string up to a pre-advance limit, wherein the reserved slop region has a length determined as a function of to the pre-advance limit.
5. The compute device of claim 4 , wherein the length of the reserved slop region equals one pointer for each position of the pre-advance limit.
6. The compute device of claim 1 , wherein the compare engine is further to compare a plurality of instances and retire the comparisons in a natural order.
7. The compute device of claim 6 , wherein the natural order is a function of an input byte order.
8. The compute device of claim 6 , wherein the natural order is further a function of one or more heuristic matching rules.
9. The compute device of claim 1 , further comprising a table updater to generate a hash of an n-byte prefix string of the current string.
10. The compute device of claim 9 , wherein the table updater is further to:
look up a hash table entry in a hash table using the hash, wherein the hash table entry holds one or more pointers to the one or more prior instances;
determine whether the hash table entry is full; and
write the hash table entry to a spill table in response to a determination that the hash table entry is full.
11. The compute device of claim 10 , wherein the table updater is to overwrite an oldest spill table entry in the spill table using the hash table entry.
12. A method for data compression, the method comprising:
receiving, by a compare engine of a computing device, a current string to be matched to one or more prior instances of the current string, the current string located within an input buffer and the one or more prior instances located within a history buffer;
identifying, by the compare engine, a limited subset of an index memory of the compare engine designated for storing pointers to the one or more prior instances, further comprising identifying a reserved slop region in the index memory; and
comparing, by the compare engine, the current string to at least one prior instance of the one or more prior instances, wherein comparing the current string further comprises locating the at least one prior instance using at least one pointer to the at least one prior instance, wherein the at least one pointer is stored within the limited subset of the index memory, and wherein comparing the current string includes prohibiting use of any pointers stored in the reserved slop region of the index memory.
13. The method of claim 12 , wherein the compare engine comprises a hardware accelerator.
14. The method of claim 12 , wherein comparing the current string comprises traversing in an order that is a reverse of an initialization order and stopping the traverse in response to entering the reserved slop region.
15. The method of claim 12 , further comprising indexing, by the computing device, the input buffer at positions from the current string up to a pre-advance limit, wherein the reserved slop region has a length determined as a function of to the pre-advance limit.
16. The method of claim 15 , wherein the length of the reserved slop region equals one pointer for each position of the pre-advance limit.
17. The method of claim 12 , wherein comparing the current string to the at least one prior instance comprises comparing the current string to a plurality of instances, the method further comprising retiring, by the compare engine, the plurality of comparisons in a natural order.
18. The method of claim 12 , further comprising generating, by a table updater of the computing device, a hash of an n-byte prefix string of the current string.
19. One or more non-transitory computer-readable storage media comprising a plurality of instructions stored thereon that, when executed by a computing device cause the computing device to:
receive, by a compare engine of the computing device, a current string to be matched to one or more prior instances of the current string, the current string located within an input buffer and the one or more prior instances located within a history buffer;
identify, by the compare engine, a limited subset of an index memory of the compare engine designated for storing pointers to the one or more prior instances, further comprising identifying a reserved slop region in the index memory; and
compare, by the compare engine, the current string to at least one prior instance of the one or more prior instances, wherein comparing the current string further comprises locating the at least one prior instance using at least one pointer to the at least one prior instance, wherein the at least one pointer is stored within the limited subset of the index memory, and wherein comparing the current string includes prohibiting use of any pointers stored in the reserved slop region of the index memory.
20. The one or more non-transitory computer-readable storage media of claim 19 , wherein the compare engine comprises a hardware accelerator.
21. The one or more non-transitory computer-readable storage media of claim 19 , wherein to compare the current string comprises traversing in an order that is a reverse of an initialization order and to stop the traverse in response to entering the reserved slop region.
22. The one or more non-transitory computer-readable storage media of claim 19 , further comprising a plurality of instructions stored thereon that, when executed by the computing device cause the computing device to index the input buffer at positions from the current string up to a pre-advance limit, wherein the reserved slop region has a length determined as a function of to the pre-advance limit.
23. The one or more non-transitory computer-readable storage media of claim 22 , wherein the length of the reserved slop region equals one pointer for each position of the pre-advance limit.
24. The one or more non-transitory computer-readable storage media of claim 19 , wherein to compare the current string to the at least one prior instance comprises to compare the current string to a plurality of instances, the computer-readable storage media further comprising a plurality of instructions stored thereon that, when executed by the computing device cause the computing device to retire, by the compare engine, the plurality of comparisons in a natural order.
25. The one or more non-transitory computer-readable storage media of claim 19 , further comprising a plurality of instructions stored thereon that, when executed by the computing device cause the computing device to generate, by a table updater of the computing device, a hash of an n-byte prefix string of the current string.