System and method for hierarchical compression

1. A method for compressing data over a link, the method comprising:

receiving a first compressed data stream and a second compressed data stream at a network node, the second compressed data stream being generated through higher-layer compression of a data stream at an upstream node;

identifying a first occurrence of a data block in the first compressed data stream;

performing link-layer compression on at least the second compressed data stream to obtain a compressed traffic flow, wherein performing link-layer compression on at least the second compressed data stream comprises identifying a second occurrence of the data block in the second compressed data stream, and replacing the second occurrence of the data block in the second compressed data stream with a link-layer compression symbol associated with the data block, the first compressed data stream being a different data stream than the second compressed data stream; and

forwarding the compressed traffic flow from the network node to a next hop node on the link.

2. The method of claim 1 , wherein the link-layer compression symbol is an identifier associated with the data block in a history table.

3. The method of claim 1 , wherein performing link-layer compression on at least the second compressed data stream to obtain a compressed traffic flow comprises:

performing link-layer compression on both the first compressed data stream and the second compressed data stream to obtain the compressed traffic flow.

4. The method of claim 3 , wherein the link-layer compression symbol is a backwards or forwards reference identifier that identifies a position of the first occurrence of the data block in the first compressed data stream.

5. The method of claim 3 , wherein performing link-layer compression on both the first compressed data stream and the second compressed data stream to obtain the compressed traffic flow comprises:

de-correlating compressed data in the second compressed data stream with compressed data in the first compressed data stream to reduce inter-stream redundancy between the second compressed data stream and the first compressed data stream.

6. The method of claim 5 , wherein the first compressed data stream and the second compressed data stream are addressed to different remote nodes.

7. The method of claim 5 , wherein the first compressed data stream and the second compressed data stream originate from different source nodes.

8. The method of claim 1 , wherein the compressed traffic flow is configured to be decompressed through link-layer de-compression at a downstream network node positioned on the link, and

wherein the second compressed data stream is configured to be decompressed through higher-layer de-compression at a remote node, the next hop node being positioned in-between the network node and the remote node.

9. The method of claim 1 , wherein identifying the second occurrence of the data block in the second compressed data stream comprises:

identifying higher-layer compression symbols in the second compressed data stream, the higher-layer compression symbols having been inserted into the second compressed data stream by the upstream node during higher-layer compression;

identifying a data chunk as a string of data positioned in-between a first higher-layer compression symbol and a second higher-layer compression symbol; and

determining that the data chunk is associated with the second occurrence of the data block.

10. The method of claim 9 , wherein the link-layer compression symbol includes an indicator that differentiates the link-layer compression symbol from the higher-layer compression symbols.

11. The method of claim 1 , wherein the second occurrence of the data block includes an intermediate higher-layer compression symbol.

12. The method of claim 11 , wherein replacing the second occurrence of the data block in the second compressed data stream with a link-layer compression symbol associated with the data block comprises:

embedding the intermediate higher-layer compression symbol within the link-layer compression symbol, the embedded intermediate higher-layer compression symbol being associated with a position of the intermediate higher-layer compression symbol within the second occurrence of the data block; and

replacing the second occurrence of the data block with the link-layer compression symbol.

13. An apparatus comprising:

a processor; and

a non-transitory computer readable storage medium storing programming for execution by the processor, the programming including instructions to:

receive a first compressed data stream and a second compressed data stream at a network node, the second compressed data stream having been generated through higher-layer compression of a data stream at an upstream node;

identify a first occurrence of a data block in the first compressed data stream;

perform link-layer compression on at least the second compressed data stream to obtain a compressed traffic flow, wherein the instructions to perform link-layer compression on at least the second compressed data stream include instructions to identify a second occurrence of the data block in the second compressed data stream, and to replace the second occurrence of the data block in the second compressed data stream with a link-layer compression symbol associated with the data block, the first compressed data stream being a different data stream than the second compressed data stream; and

forward the compressed traffic flow from the network node to a next hop node on the link.

14. The apparatus of claim 13 , wherein the instructions to perform link-layer compression on at least the second compressed data stream to obtain a compressed traffic flow include instructions to:

perform link-layer compression on both the first compressed data stream and the second compressed data stream to obtain the compressed traffic flow, wherein the link-layer compression symbol is a backwards or forwards reference identifier that identifies a position of the first occurrence of the data block in the first compressed data stream.

15. The apparatus of claim 13 , wherein the instructions to perform link-layer compression on at least the second compressed data stream to obtain a compressed traffic flow include instructions to:

perform link-layer compression on both the first compressed data stream and the second compressed data stream to obtain the compressed traffic flow by de-correlating compressed data in the second compressed data stream with compressed data in the first compressed data stream to reduce inter-stream redundancy between the second compressed data stream and the first compressed data stream.

16. The apparatus of claim 15 , wherein the first compressed data stream and the second compressed data stream are addressed to different remote nodes.

17. The apparatus of claim 15 , wherein the first compressed data stream and the second compressed data stream originate from different source nodes.

18. The apparatus of claim 13 , wherein the compressed traffic flow is configured to be decompressed through link-layer de-compression at a downstream network node positioned on the link, and

wherein the second compressed data stream is configured to be decompressed through higher-layer de-compression at a remote node, the next hop node being positioned in-between the network node and the remote node.

19. The apparatus of claim 13 , wherein the instructions to identify the second occurrence of the data block in the second compressed data stream include instructions to:

identify higher-layer compression symbols in the second compressed data stream, the higher-layer compression symbols having been inserted into the second compressed data stream by the upstream node during higher-layer compression;

identify a data chunk as a string of data positioned in-between a first higher-layer compression symbol and a second higher-layer compression symbol; and

determine that the data chunk is associated with the second occurrence of the data block.

20. The apparatus of claim 19 , wherein the link-layer compression symbol includes an indicator that differentiates the link-layer compression symbol from the higher-layer compression symbols.

21. The apparatus of claim 13 , wherein the second occurrence of the data block includes an intermediate higher-layer compression symbol.

22. The apparatus of claim 21 , wherein the instructions to replace the second occurrence of the data block in the second compressed data stream with a link-layer compression symbol associated with the data block include instructions to:

embed the intermediate higher-layer compression symbol within the link-layer compression symbol, the embedded intermediate higher-layer compression symbol being associated with a position of the intermediate higher-layer compression symbol within the second occurrence of the data block; and

replace the second occurrence of the data block with the link-layer compression symbol.

23. A method for link-layer de-compression, the method comprising:

receiving, by a network node, a compressed traffic flow from an upstream network node, the compressed traffic flow carrying compressed data that includes at least a first link-layer compression symbol and a first higher-layer compression symbol; and

performing, by the network node, link-layer de-compression on the compressed traffic flow to obtain a compressed data stream, wherein performing link-layer de-compression comprises determining that the first link-layer compression symbol is associated with a first data block, and replacing the link-layer compression symbol with the first data block, wherein the higher-layer compression symbol remains in the compressed data after the link-layer de-compression.

24. The method of claim 23 , wherein the link-layer compression symbol was inserted into the compressed data during link-layer compression at an upstream network node, and wherein the higher-layer compression symbol was inserted into the compressed data during higher-layer compression at a source node.

25. The method of claim 24 , further comprising:

forwarding the compressed data stream to a remote node.

26. The method of claim 24 , further comprising:

performing, by the network node, link-layer re-compression on at least the compressed data stream to obtain a new compressed traffic flow, wherein performing link-layer re-compression on at least the compressed data stream comprises identifying a redundant occurrence of a second data block in the compressed data stream, and replacing the redundant occurrence of the second data block with a second link-layer compression symbol associated with the second data block; and

forwarding the new compressed traffic flow to a next hop node.

27. The method of claim 26 , wherein performing link-layer compression on at least the compressed data stream further includes re-substituting the first link-layer compression symbol for the first data block.

28. The method of claim 26 , wherein the redundant occurrence of the second data block includes a data string encompassing the first data block, and wherein performing link-layer compression on at least the compressed data stream includes inserting the second link-layer compression symbol into the compressed data without inserting the first link-layer compression symbol into the compressed data.

29. The method of claim 28 , wherein the higher-layer compression is lossless, and wherein the link-layer compression is lossy.

30. An apparatus comprising:

a processor; and

a non-transitory computer readable storage medium storing programming for execution by the processor, the programming including instructions to:

receive a compressed traffic flow from an upstream network node, the compressed traffic flow carrying compressed data that includes at least a first link-layer compression symbol and a first higher-layer compression symbol; and

perform link-layer de-compression on the compressed traffic flow to obtain a compressed data stream, wherein the instructions to perform link-layer de-compression include instructions to determine that the first link-layer compression symbol is associated with a first data block, and replacing the link-layer compression symbol with the first data block, wherein the higher-layer compression symbol remains in the compressed data after the link-layer de-compression.

31. A method for compressing data over a link, the method comprising:

receiving, by a network node, a compressed data stream, the compressed data stream having been generated through higher-layer compression of a data stream at an upstream node;

identifying higher layer compression symbols in the compressed data stream;

performing data chunking in-between the higher layer compression symbols during link-layer compression to obtain one or more data chunks, wherein the data chunks and at least a first one of the higher layer compression symbols are positioned in a compressed data block of the compressed data stream;

determining that the one or more data chunks match known data chunks of a known data block;

comparing the compressed data block with the known data block to determine if the compressed data block matches the known data block;

substituting a link-layer compression symbol for the compressed data block if the compressed data block matches the known data block, thereby obtaining a compressed traffic flow; and

forwarding the compressed traffic flow from the network node to a next hop node on the link.

32. The method of claim 31 , wherein substituting the link-layer compression symbol for the compressed data block comprises:

embedding the first higher layer compression symbol in the link-layer compression symbol; and

replacing the compressed data block with the link-layer compression symbol.

33. A network node comprising:

a processor; and

a non-transitory computer readable storage medium storing programming for execution by the processor, the programming including instructions to:

receive a compressed data stream, the compressed data stream having been generated through higher-layer compression of a data stream at an upstream node;

identify higher layer compression symbols in the compressed data stream;

perform data chunking in-between the higher layer compression symbols during link-layer compression to obtain one or more data chunks, wherein the data chunks and at least a first one of the higher layer compression symbols are positioned in a compressed data block of the compressed data stream;

determine that the one or more data chunks match known data chunks of a known data block;

compare the compressed data block with the known data block to determine if the compressed data block matches the known data block;

substitute a link-layer compression symbol for the compressed data block if the compressed data block matches the known data block, thereby obtaining a compressed traffic flow; and

forward the compressed traffic flow from the network node to a next hop node on the link.