Network traffic synchronization and data compression in redundant network topologies
View Patent ↗Synchronization of network traffic compression mechanisms deployed in redundant network topologies. In one implementation, the present invention features the synchronization of compression statistics on redundant network devices to facilitate failover and load sharing operations in the management of data flows traversing computer network environments. In one implementation, compression meta data is appended to synchronization packets and transmitted to one or more partner or redundant network devices. The receiving network devices use the compression meta data to synchronize one or more data flow control processes or data structures. Implementations of the present invention also feature process flows that increase the efficiency of synchronizing compression related operations.
1. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
at least one network interface;
a compression module operative to
compress outgoing packets;
de-compress incoming packets; and
compute compression statistics related to the compressed and de-compressed packets, wherein the compression statistics includes at least a packet compression ratio or a packet size difference before and after compression;
a packet processor operative to
receive packets on the at least one network interface;
receive synchronization packets from at least one partner network device, wherein at least one synchronization packet includes compression statistics computed by the partner network device;
transmit the packets received on the at least one network interface as synchronization packets to the partner network device,
wherein at least one synchronization packet transmitted on the at least one network interface includes a compression statistic computed by the compression module.
2. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
at least one network interface;
a compression module operative to
compress outgoing packets;
de-compress incoming packets; and
compute compression statistics related to the compressed and de-compressed packets;
a packet processor operative to
receive packets on the at least one network interface;
receive synchronization packets from at least one partner network device, wherein at least one synchronization packet includes compression statistics computed by the partner network device;
transmit the packets received on the at least one network interface as synchronization packets to the partner network device;
wherein at least one synchronization packet transmitted on the at least one network interface includes a compression statistic computed by the compression module; and
a flow control module operative to allocate bandwidth to packets based at least in part on the compression statistics associated with the packets.
3. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
at least one network interface;
a compression module operative to
compress outgoing packets;
de-compress incoming packets; and
compute compression statistics related to the compressed and de-compressed packets; and
a packet processor operative to
receive packets on the at least one network interface;
receive synchronization packets from at least one partner network device, wherein at least one synchronization packet includes compression statistics computed by the partner network device;
transmit the packets received on the at least one network interface as synchronization packets to the partner network device, wherein at least one synchronization packet transmitted on the at least one network interface includes a compression statistic computed by the compression module; and
discard the received synchronization packets.
4. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
a first network interface for communication with a first network device;
a second network interface for communication with a second network device;
at least a third network interface for exchange of synchronization packets with a partner network device;
a compression module operative to
compress outbound packets received at the first network interface;
de-compress inbound packets received at the second network interface; and
compute compression statistics related to the compressed and de-compressed packets;
a packet processor operative to
receive packets on the first and second network interfaces, wherein the data packets are addressed to destination hosts;
receive synchronization packets on the at least a third network interface, wherein at least one synchronization packet includes compression statistics computed by a partner network device;
transmit, on the at least a third network interface, the packets received on the first network interface as synchronization packets to the partner network device,
transmit, on the at least a third network interface, the packets received on the second network interface as synchronization packets to the partner network device;
wherein at least one synchronization packet transmitted on the at least a third network interface includes a compression statistic computed by the compression module; and
a flow control module operative to allocate bandwidth to packets based at least in part on the compression statistics associated with the packets.
5. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
a first network interface for communication with a first network device;
a second network interface for communication with a second network device;
at least a third network interface for exchange of synchronization packets with a partner network device;
a compression module operative to
compress outbound packets received at the first network interface;
de-compress inbound packets received at the second network interface; and
compute compression statistics related to the compressed and de-compressed packets;
a packet processor operative to
receive packets on the first and second network interfaces, wherein the packets are addressed to destination hosts;
receive synchronization packets on the at least a third network interface, wherein at least one synchronization packet includes compression statistics computed by a partner network device;
transmit, on the at least a third network interface, the packets received on the first network interface as synchronization packets to the partner network device; and
transmit, on the at least a third network interface, the packets received on the second network interface as synchronization packets to the partner network device, wherein at least one synchronization packet transmitted on the at least a third network interface includes a compression statistic computed by the compression module;
a flow control module operative to allocate bandwidth to packets based at least in part on the compression statistics associated with the packets; and
a tunnel probing module operative to
dynamically probe a communications path to a destination host for at least one network device having compression capabilities; and
establish a compression tunnel with an identified network device in the communications path.
6. The apparatus of claim 5 wherein the tunnel probing module is operative to
transmit probe requests along the communications path to the destination host; and
receive probe responses from network devices in the communications path to the destination host.
7. The apparatus of claim 6 wherein at least one probe response identifies a partner network device; and wherein the tunnel probing module is operative to establish a back up tunnel with the partner network device.
8. The apparatus of claim 6 wherein the tunnel probing module is further operative to
receive a probe request from a network device in the communications path between a source host and a destination host;
respond to the probe request by transmitting a probe response;
copy the probe request to a partner network device.
9. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
a first network interface for communication with a first network device;
a second network interface for communication with a second network device;
at least a third network interface for exchange of synchronization packets with a partner network device;
a compression module operative to
compress outbound packets received at the first network interface;
de-compress inbound packets received at the second network interface; and
compute compression statistics related to the compressed and de-compressed packets;
a packet processor operative to
receive packets on the first and second network interfaces, wherein the packets are addressed to destination hosts;
receive synchronization packets on the at least a third network interface, wherein at least one synchronization packet includes compression statistics computed by a partner network device;
discard the received synchronization packets;
transmit, on the at least a third network interface, the packets received on the first network interface as synchronization packets to the partner network device; and
transmit, on the at least a third network interface, the packets received on the second network interface as synchronization packets to the partner network device;
wherein at least one synchronization packet transmitted on the at least a third network interface includes a compression statistic computed by the compression module; and
a flow control module operative to allocate bandwidth to packets based at least in part on the compression statistics associated with the packets.
10. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
a first network interface for communication with a first network device;
a second network interface for communication with a second network device;
a third network interface for exchange of synchronization packets with a partner network device;
a fourth network interface for exchange of synchronization packets with the partner network device;
a compression module operative to
compress outbound packets received at the first network interface;
de-compress inbound packets received at the second network interface; and
compute compression statistics related to the compressed and de-compressed packets;
a packet processor operative to
receive packets on the first and second network interfaces, wherein the packets are addressed to destination hosts;
receive synchronization packets on the at least a third network interface, wherein at least one synchronization packet includes compression statistics computed by a partner network device;
transmit, on the at least a third network interface, the packets received on the first network interface as synchronization packets to the partner network device;
transmit, on the at least a third network interface, the packets received on the second network interface as synchronization packets to the partner network device;
transmit, on the third network interface, the data packets received on the first network interface as synchronization packets to the partner network device; and
transmit, on the fourth network interface, the data packets received on the second network interface as synchronization packets to the partner network device;
wherein at least one synchronization packet transmitted on the at least a third network interface includes a compression statistic computed by the compression module; and
a flow control module operative to allocate bandwidth to packets based at least in part on the compression statistics associated with the packets.
11. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
a first network interface for communication with a first network device;
a second network interface for communication with a second network device;
at least a third network interface for exchange of synchronization packets with a partner network device;
a compression module operative to
compress outbound packets received at the first network interface;
de-compress inbound packets received at the second network interface; and
compute compression statistics related to the compressed and de-compressed packets;
a packet processor operative to
receive packets on the first and second network interfaces, wherein the packets are addressed to destination hosts;
receive synchronization packets on the at least a third network interface, wherein at least one synchronization packet includes compression statistics computed by a partner network device;
encapsulate the packets received on the first and second network interfaces with synchronization headers before transmitting the packets on the at least a third network interface;
transmit, on the at least a third network interface, the packets received on the first network interface as synchronization packets to the partner network device;
transmit, on the at least a third network interface, the packets received on the second network interface as synchronization packets to the partner network device; and
decapsulate the synchronization packets received on the at least a third network interface network interface;
wherein at least one synchronization packet transmitted on the at least a third network interface includes a compression statistic computed by the compression module; and
a flow control module operative to allocate bandwidth to packets based at least in part on the compression statistics associated with the packets.
12. The apparatus of claim 11 wherein the control module is further operative to
include meta information relating to the data packets in the synchronization header.
13. The apparatus of claim 4 wherein the first network interface is a wired network interface.
14. The apparatus of claim 4 wherein the first network interface is a wireless network interface.
15. The apparatus of claim 4 wherein the at least a third network interface is a wireless network interface.
16. The apparatus of claim 8 wherein the fourth network interface is a wireless network interface.
17. The apparatus of claim 4 wherein the packet processor is further operative to verify the packets received on the at least a third network interface.
18. The apparatus of claim 17 wherein the synchronization packets include a magic identifier, and wherein the packet processor is operative to verify the packets received on the at least a third network interface by validating the magic identifier.
19. The apparatus of claim 17 wherein the control module is further operative to include magic identifiers to synchronization packets transmitted on the at least a third network interface.
20. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
a first network interface for communication with a first network device;
a second network interface for communication with a second network device;
at least a third network interface for exchange of synchronization packets with a partner network device;
a compression module operative to
compress outbound packets received at the first network interface;
de-compress inbound packets received at the second network interface; and
compute compression statistics related to the compressed and de-compressed packets;
a packet processor operative to
receive packets on the first and second network interfaces, wherein the data packets are addressed to destination hosts;
receive synchronization packets on the at least a third network interface, wherein at least one synchronization packet includes compression statistics computed by a partner network device;
transmit, on the at least a third network interface, the packets received on the first network interface as synchronization packets to the partner network device;
transmit, on the at least a third network interface, the packets received on the second network interface as synchronization packets to the partner network device, wherein at least one synchronization packet transmitted on the at least a third network interface includes a compression statistic computed by the compression module; and
compose summary versions of the packets received on the first and second network interfaces and transmit the summary versions as synchronization packets on the at least a third network interface; and
a flow control module operative to allocate bandwidth to packets based at least in part on the compression statistics associated with the packets.
21. The apparatus of claim 20 wherein the summary versions each comprise a packet header and a payload size.
22. The apparatus of claim 4 wherein the compression statistics comprise a compression ratio.
23. The apparatus of claim 4 wherein the compression statistics comprise the size difference between at least one uncompressed packet and the corresponding compressed packet.
24. The apparatus of claim 4 wherein the compression statistics comprise the size difference between at least one compressed packet and the corresponding de-compressed packet.
25. In a computer network environment including a first network comprising a plurality of remote hosts and at least first and second redundant compression devices disposed in the communications paths between the remote hosts and a second network, a method comprising
identifying at least first and second redundant compression devices in communications paths between a second network and the remote hosts of the first network;
tunneling data flows associated with a given remote host to a selected compression device from the identified redundant compression devices;
probing the communications paths to a first remote host of the first network for the at least first and second redundant compression devices; and
selecting, for the first remote host, one of the at least first and second redundant compression devices based on the results of the probing step.
26. The method of claim 25 further comprising
repeating the probing and selecting step for a desired number of remote hosts of the first network.
27. The method of claim 25 further comprising
selecting, for at least a second remote host of the first network, one of the at least first and second redundant compression devices based on the results of the probing step.
28. The method of claim 25 wherein the probing step comprises
transmitting probe requests to the remote hosts; and
receiving probe responses from one of the at least first and second redundant compression devices in the communications paths to the remote hosts.
29. The method of claim 28 wherein at least one probe response identifies a partner compression device.
30. A method facilitating the synchronization of network traffic with at least one compression-enabled network device, comprising
receiving, at a network device, tunnel packets addressed to the network device on a first network interface;
receiving, at the network device, packets addressed to destination hosts on the second network interface;
receiving synchronization packets from at least one partner network device;
decompressing tunnel packets received at the first network interface and transmitting the decompressed data packets received on the first network interface as synchronization packets to at least one partner network device, wherein the synchronization packet includes compressing statistics relating to the decompressing step;
transmitting the packets received on the second network interface as synchronization packets to the at least one partner network device;
processing the packets received on the first and second network interfaces, and the synchronization packets; and
discarding the synchronization packets.
31. The method of claim 30 further comprising
compressing outbound tunnel packets received at the second network interface and transmitting them from the first network interface.
32. The method of claim 30 further comprising
transmitting, after the decompressing step, tunnel packets received at the first network interface from the second network interface.
33. The method of claim 30 wherein the synchronization packets are transmitted and received on the first network interface.
34. The method of claim 30 wherein the synchronization packets are transmitted and received on at least a third network interface.
35. The method of claim 30 wherein the synchronization packets include a VLAN identifier associated with the network interface on which the corresponding packet was received.
36. The method of claim 30 wherein the synchronization packets include an MPLS tag identifying the network interface on which the corresponding packet was received.
37. An apparatus enabling synchronization of network traffic with at least one compression-enabled network device, comprising
a first network interface for communication with a first network device;
a second network interface for communication with a second network device;
at least a third network interface for exchange of synchronization packets with a partner network device;
a packet processor operative to
process data packets on the first and second network interfaces, wherein the data packets are addressed to destination hosts;
process synchronization packets on the at least a third network interface, wherein at least one synchronization packet includes compression statistics computed by a partner network device;
transmit, on the at least a third network interface, the data packets received on the first network interface as synchronization packets to the partner network device,
transmit, on the at least a third network interface, the data packets received on the second network interface as synchronization packets to the partner network device; and
a compression module operative to
compress outbound data packets received at the first network interface and outbound synchronization packets received at the at least a third network interface;
de-compress inbound data packets received at the second network interface;
compute compression statistics related to the compressed and de-compressed packets; and
maintain a shadow dictionary for compression operations associated with synchronization packets received at the at least a third network interface; and
a flow control module operative to allocate bandwidth to packets based at least in part on the compression statistics associated with the packets.
38. The apparatus of claim 37 wherein the compression module is operative to de-compress inbound synchronization packets received at the at least a third network interface.
39. The apparatus of claim 37 wherein the apparatus is operative to decompress inbound data packets received at the second network interface and transmit the decompressed inbound data packets to the partner network device.
40. An apparatus enabling synchronization of network traffic with at least one compression-enabled partner network device, comprising
a first network interface;
a second network interface;
at least a third network interface for exchange of synchronization packets with a partner network device;
a compression module operative to
maintain a compression tunnel with a compression-enabled network device;
compute inbound compression statistics for inbound packets;
compute outbound compression statistics for outbound packets; and
a packet processor operative to
receive inbound packets on the first network interface,
identify compressed inbound packets received on the first network interface and pass the compressed inbound packets to the compression module;
receive decompressed inbound packets from the compression module;
transmit, on the at least a third network interface, decompressed inbound data packets received on the first network interface as inbound synchronization packets to the partner network device, wherein the inbound synchronization packets include inbound compression statistics computed by the compression module;
receive outbound packets on the second network interface,
identify outbound packets corresponding to a compression tunnel and pass the identified outbound packets to the compression module;
transmit, on the at least a third network interface, the outbound packets received on the second network interface as outbound synchronization packets to the partner network device, wherein the outbound synchronization packets include outbound compression statistics computed by the compression module;
receive inbound synchronization packets on the at least a third network interface from at least one partner network device, wherein at least one of the inbound synchronization packets include an inbound compression statistic computed by a partner network device; and
receive outbound synchronization packets on the at least a third network interface from the at least on partner network device, wherein at least one of the outbound synchronization packets include an outbound compression statistic computed by a partner network device.
41. The apparatus of claim 40 further comprising
a flow control module operative to
allocate inbound bandwidth to inbound packets and inbound synchronization packets based in part on the inbound compression statistics corresponding to the inbound packets and inbound synchronization packets; and
allocate outbound bandwidth to outbound packets and outbound synchronization packets based in part on the outbound compression statistics corresponding to the outbound packets and outbound synchronization packets.
42. The apparatus of claim 40 wherein the inbound compression statistics characterize the difference between at least one compressed packet and the corresponding decompressed packet.
43. The apparatus of claim 40 wherein the inbound compression statistics are compression ratios characterizing the difference between at least one compressed inbound packet and the corresponding decompressed inbound packet.
44. The apparatus of claim 40 wherein the outbound compression statistics are, relative to a given outbound packet, an estimated compression statistic.
45. The apparatus of claim 40 wherein the outbound compression statistics are weighted moving averages of the compression ratio in the outbound direction for each data flow.
46. The apparatus of claim 40 wherein the outbound compression statistics are packet fix up values characterizing the difference between the estimated compressed size of an outbound packet and the actually compressed size of the outbound packet.
47. The apparatus of claim 40 wherein the outbound compression statistics are packet fix up values characterizing the difference between the estimated compressed size of a previous outbound packet and the actually compressed size of the previous outbound packet.
48. The apparatus of claim 47 wherein the outbound compression statistics further include the estimated compressed size of a corresponding outbound packet.
49. The apparatus of claim 1 wherein the compression statistic included in a synchronization packet characterizes compression or de-compression of a packet corresponding to the synchronization packet.
50. The apparatus of claim 4 wherein the compression statistic included in a synchronization packet characterizes compression or de-compression of a packet corresponding to the synchronization packet.