Compressing packet data
A system, method, and computer program for compressing packet data is provided. In exemplary embodiments, one or more blocks may be identified that include block data similar to packet data of one or more packets. The one or more blocks may comprise archives of previously transferred packets. The packet data may be compressed based, at least partially, on the block data. Accordingly, the compressed packet data may be transferred over a communication network.
1. A method for compressing packets, comprising:
identifying block data from one or more blocks that include data similar to packet data from one or more packets, the block data comprising archives of previously transferred data;
associating the packet data with the identified block data that includes data similar to the packet data and using Lempel-Ziv (LZ) based encoding on the identified block data and the packet data that has been associated with the identified block data to generate compressed packet data, wherein the encoding and compressing is based at least in part on the identified block data and wherein the compressed packet data comprises a block indicator and an encoded portion comprising at least one length-distance pair, a distance parameter of the length-distance pair defining a byte position within the associated block data; and
transferring the compressed packet data over a communication network.
2. The method of claim 1 , further comprising intercepting the one or more packets after the one or more packets are sent from a computing device.
3. The method of claim 1 , wherein the one or more blocks are stored in a network memory.
4. The method of claim 1 , further comprising dividing the one or more blocks into sub-blocks.
5. The method of claim 1 , wherein the compressing comprises identifying one or more portions of the packet data that match one or more portions of the block data.
6. The method of claim 1 , further comprising generating one or more data structures associated with the packet data and the block data.
7. The method of claim 6 , wherein the one or more data structures are stored in a cache.
8. The method of claim 1 , wherein the compressing is further based on identifying similar data within the packet data itself.
9. The method of claim 1 , further comprising building a dictionary based on the one or more blocks.
10. The method of claim 1 , further comprising:
receiving the compressed packet data;
retrieving the one or more blocks based at least in part on the block indicator in the compressed packet data; and
decompressing the compressed packet data based at least in part on the block data from the one or more blocks and the at least one length-distance pair in the compressed packet data.
11. The method of claim 10 , wherein the decompressing comprises Lempel-Ziv (LZ) based decoding.
12. A system for compressing packets, comprising:
a network memory module executable by a processor and configured to store blocks in a memory, the blocks comprising archives of previously transferred data;
a compression-decompression engine configured to identify block data from one or more blocks that include data similar to packet data from one or more packets, and to associate the packet data with the identified block data that includes data similar to the packet data and use Lempel-Ziv (LZ) based encoding on the identified block data and the packet data that has been associated with the identified block data to generate compressed packet data, wherein the encoding and compressing is based at least in part on the identified block data and wherein the compressed packet data comprises a block indicator and an encoded portion comprising at least one length-distance pair, a distance parameter of the length-distance pair defining a byte position within the associated block data; and
an interface module configured to transfer the compressed packet data over a communication network.
13. The system of claim 12 , wherein the interface module is further configured to intercept the one or more packets after the one or more packets are sent from a computer.
14. The system of claim 12 , wherein the compression-decompression engine is further configured to divide the one or more blocks into sub-blocks.
15. The system of claim 12 , wherein the compression-decompression engine comprises a scan module configured to scan the packet data to generate data structures associated with the packet data.
16. The system of claim 15 , wherein the scan module is further configured to generate one or more data structures associated with the packet data and the block data.
17. The system of claim 12 , wherein the compression-decompression engine further comprises a map module configured to map portions of the packet data to locations with similar data within the block data and determine at least one length-distance pair comprising a length of each portion mapped and a distance from each of the portions in the packet data to the mapped location in the block data.
18. The system of claim 12 , wherein the compression-decompression engine comprises an encoding-decoding module configured to perform LZ based encoding and LZ based decoding.
19. The system of claim 12 , wherein the compression-decompression engine is further configured to compress the packet data based on identifying similar data within the packet data itself.
20. The system of claim 12 , wherein the interface module is further configured to receive compressed packet data; and wherein the compression-decompression engine is further configured to retrieve the one or more blocks based at least in part on the block indicator in the compressed packet data and to decompress the compressed packet data based at least in part on the block data from the one or more blocks and the at least one length-distance pair in the compressed packet data.
21. A non-transitory machine readable medium having embodied thereon a program, the program providing instructions for a method for compressing packets, the method comprising:
identifying block data from one or more blocks that include data similar to packet data from one or more packets, the block data comprising archives of previously transferred data;
associating the packet data with the identified block data that includes data similar to the packet data and using Lempel-Ziv (LZ) based encoding on the identified block data and the packet data that has been associated with the identified block data to generate compressed packet data, wherein the encoding and compressing is based at least in part on the identified block data and wherein the compressed packet data comprises a block indicator and an encoded portion comprising at least one length-distance pair, a distance parameter of the length-distance pair defining a byte position within the associated block data; and
transferring the compressed packet data over a communication network.