Dynamic scaling of redundancy elimination middleboxes

1. A method for dynamic scaling of redundancy elimination middleboxes comprising an encoding middlebox and a decoding middlebox in a communication network, the method comprising:

determining a load of incoming data at the encoding middlebox in the communication network; and

modifying a number of encoder instances in the encoding middlebox and a number of decoder instances in a decoding middlebox based on the load of incoming data, the modifying including configuring a classifier to define a new class corresponding to each of one or more new encoder instances and modifying an ordered list of each of the encoder instances present in the encoding middlebox before addition of the one or more new encoder instances, the ordered list relating the new class to each of the encoder instances; and

handling failure recovery of at least one of the classifier, the encoder instances, the decoder instances, and a merger, the handling including recovering packet loss based on at least one of a decoder feedback, a transmission control packet sequence number, and a reliable transport.

2. The method as claimed in claim 1 , wherein the modifying further comprises adding one or more new encoder instances on identifying the load of incoming data greater than a predefined limit, wherein the adding comprises: and

distributing packets of data across all the encoder instances present in the encoding middlebox.

3. The method as claimed in claim 1 , wherein the modifying further comprises adding one or more new encoder instances on identifying the load of incoming data greater than a predefined limit, wherein the adding comprises:

assigning a new hash-range to each of the encoder instances based on number of the one or more new encoding instances, wherein the new hash-range and a old hash-range are stored in each of the encoder instances;

storing chunks of packets in content stores of the encoder instances based on the new hash-ranges;

discarding the old hash-range of each of the encoder instances when old chunks are evicted from the content stores; and

distributing the chunks of packets across the encoder instances left in the encoding middlebox.

4. The method as claimed in claim 1 , wherein the modifying further comprises removing one or more existing encoder instances on identifying the load of incoming data less than a predefined limit, wherein the removing comprises:

discarding the ordered list of each of the one or more existing encoder instances, wherein each of the one or more existing encoder instances perform lookups and do not store any new content;

removing each of the one or more existing encoder instances upon identifying content of the one or more existing encoder instances obsolete; and

distributing packets of data across all encoder instances present in the encoding middlebox.

5. The method as claimed in claim 1 , wherein the modifying further comprises removing one or more existing encoder instances on identifying the load of incoming data less than a predefined limit, wherein the removing comprises:

discarding the ordered list of each of the one or more existing encoder instances, wherein the one or more existing encoder instances perform lookups and do not store any new content;

distributing a hash-range of each of the one or more existing encoder instances equally among encoder instances present in the encoding middlebox;

storing chunks of packets in the content store of each of the encoder instances based on a new hash-range, wherein the new hash-range for each of the encoder instance is computed based the distributing; and

removing one or more existing encoder instances upon identifying content of the one or more existing encoder instances obsolete; and

distributing traffic across all encoder instances present in the encoding middlebox.

6. The method as claimed in claim 1 , wherein the modifying further comprises adding one or more new decoder instances on identifying the load of incoming data greater than a predefined limit, wherein the ordered list of the decoder instances is in reverse order of the ordered lists for the encoder instances, and wherein the adding comprises:

modifying an ordered list of each of decoder instances present in the decoding middlebox before addition of the one or more decoder instances, wherein the ordered list of the decoder instances is in reverse order of the ordered lists for the encoder instances; and

distributing encoded packets across all decoder instances present in the decoding middlebox for decoding.

7. The method as claimed in claim 1 , wherein the modifying further comprises removing one or more existing decoder instances on identifying the load of incoming data less than a predefined limit, wherein the removing comprises:

removing each of the one or more existing decoder instances from the decoding middlebox;

modifying an ordered list of each of decoder instances present in the decoding middlebox before removal of the one or more decoder instances, wherein the ordered list of the decoder instances is in reverse order of the ordered lists for the encoder instances; and

distributing encoded packets across all decoder instances present in the decoding middlebox for decoding.

8. The method as claimed in claim 1 further comprising handling failure recovery of at least one of a classifier, encoder instances, decoder instances, a merger, and a combination thereof.

9. A WAN optimization system for dynamic scaling of RE middleboxes in a communication network, the WAN optimization system comprising:

an encoding middlebox, wherein the encoding middlebox,

determines a load of incoming data at the encoding middlebox in the communication network, and

modifies a number of encoder instances in the encoding middlebox based on the load of incoming data, the modifying including configuring a classifier to define a new class corresponding to each of one or more new encoder instances and modifying an ordered list of each of the encoder instances present in the encoding middlebox before addition of the one or more new encoder instances, the ordered list relating the new class to each of the encoder instances,

handles failure recovery of at least one of the classifier, the encoder instances, decoder instances, and a merger, the handling including recovering packet loss based on at least one of a decoder feedback, a transmission control packet sequence number, and a reliable transport; and

a decoding middlebox, wherein the decoding middlebox,

modifies a number of decoder instances in a decoding middlebox based on the load of incoming data.

10. The WAN optimization system as claimed in claim 9 , wherein the encoding middlebox,

adds one or more new encoder instances on identifying the load of incoming data greater than a predefined limit; and

removes one or more existing encoder instances on identifying the load of incoming data less than the predefined limit.

11. The WAN optimization system as claimed in claim 9 , wherein the decoding middlebox,

adds one or more new decoder instances upon addition of the one or more new encoder instances in the encoding middlebox; and

removes one or more existing decoder instances upon removal of the one or more existing encoder instances in the encoding middlebox.

12. The WAN optimization system as claimed in claim 9 further comprising a failure recovery system for failure recovery in the RE middleboxes, wherein the failure recovery system comprises a failure recovery module to provide mechanism for failure recovery of at least one of a classifier, encoder instances, decoder instances, a merger, and a combination thereof.

13. The WAN optimization system as claimed in claim 12 , wherein the failure recovery module handles packet loss based on at least one of a decoder feedback, a transmission control protocol sequence number, and a reliable transport.

14. A non-transitory computer-readable medium having computer-executable instructions that when executed perform acts comprising:

determining a load of incoming data at an encoding middlebox in a communication network; and

modifying a number of encoder instances in the encoding middlebox and a number of decoder instances in a decoding middlebox based on a load of incoming data, the modifying including configuring a classifier to define a new class corresponding to each of one or more new encoder instances and modifying an ordered list of each of the encoder instances present in the encoding middlebox before addition of the one or more new encoder instances, the ordered list relating the new class to each of the encoder instances; and

handling failure recovery of at least one of the classifier, the encoder instances, the decoder instances, and a merger, the handling including recovering packet loss based on at least one of a decoder feedback, a transmission control packet sequence number, and a reliable transport.

15. An encoding middlebox in a communication network, wherein the encoding middlebox includes hardware having a processor configured to execute machine readable instructions to,

determine a load of incoming data at the encoding middlebox in the communication network;

modify a number of encoder instances in the encoding middlebox based on the load of incoming data, the modifying including configuring a classifier to define a new class corresponding to each of one or more new encoder instances and modifying an ordered list of each of the encoder instances present in the encoding middlebox before addition of the one or more new encoder instances, the ordered list relating the new class to each of the encoder instances; and

handle failure recovery of at least one of the classifier, the encoder instances, decoder instances, and a merger, the handling including recovering packet loss based on at least one of a decoder feedback, a transmission control packet sequence number, and a reliable transport.

16. The encoding middlebox as claimed in claim 15 , wherein the encoding middlebox encodes data packets for redundancy elimination in the communication network.

17. A decoding middlebox in a communication network, wherein the decoding middlebox includes hardware having a processor configured to execute machine readable instructions to modify a number of decoder instances in the decoding middlebox based on a number of encoding instances in an encoding middlebox, the modifying including configuring a classifier to define a new class corresponding to each of one or more new encoder instances and modifying an ordered list of each of the encoder instances present in the encoding middlebox before addition of the one or more new encoder instances, the ordered list relating the new class to each of the encoder instances, and handle failure recovery of at least one of the classifier, the encoder instances, the decoder instances, and a merger, the handling including recovering packet loss based on at least one of a decoder feedback, a transmission control packet sequence number, and a reliable transport.

18. The decoding middlebox as claimed in claim 17 , wherein the decoding middlebox decodes data packets encoded by the encoding middlebox for redundancy elimination in the communication network.