Distributed web object identification for web caching

1. A method comprising:

receiving a request for a content object at a first proxy caching (PC) engine, the request comprising a request packet;

determining whether the content object is cached at the first PC engine; and

in response to determining that the content object is not cached at the first PC engine:

modifying an element of the request packet, wherein modifying the element of the request packet comprises add an accept-encoding header in the request packet, the accept-encoding header providing an indication to an upstream PC engine that the first PC engine accepts a fingerprint associated with the content object,

forwarding the modified request packet toward a second PC engine, and

receiving a response to the forwarded request packet from the second PC engine, the response comprising the forwarded request packet with a second modification in the accept-encoding header.

2. The method of claim 1 , wherein modifying the element comprises modifying a header of the request packet.

3. The method of claim 2 , wherein modifying the header of the request packet further comprises adding the accept-encoding header to the header of the request packet.

4. The method of claim 1 , wherein initiating the accept-encoding header comprises initiating the accept-encoding header having a null value.

5. The method of claim 1 , wherein the second modification comprises a non-null value in the header.

6. The method of claim 1 , wherein the second modification comprises the fingerprint associated with the content object.

7. The method of claim 6 , further comprising:

determining whether the fingerprint is associated with a cached object; and

in response to determining that the fingerprint is associated with the cached object, responding to the request for the content object with the cached object.

8. The method of claim 1 , further comprising:

receiving a response to the forwarded request packet, wherein the response comprises a copy of the requested content object; and

responding to the request for the content object with the copy of the requested content object.

9. An apparatus comprising:

a memory storage; and

a processor coupled to the memory storage, the processor operative to:

receive a request for a content object at a first proxy caching (PC) engine, the request comprising a request packet;

determine whether the content object is associated with a cached object in the memory storage,

in response to determining that the content object is not associated with the cached object in the memory storage:

modify the request packet to add an accept-encoding header to the request packet, the accept-encoding header providing an indication to an upstream PC engine that the first PC engine accepts a fingerprint associated with the content object;

forward the modified request packet to a second PC engine, and

receive a response to the forwarded request packet from the second PC engine, the response comprising the forwarded request packet with a second modification in the accept-encoding header.

10. The apparatus of claim 9 , wherein the processor being further operative to forward the request packet toward a content provider associated with the content object.

11. The apparatus of claim 9 , wherein the processor is further operative to:

receive another response to the forwarded request packet, wherein the another response comprises a copy of the requested content object; and

respond to the request with the copy of the requested content object.

12. The apparatus of claim 9 , wherein

the response further comprises a copy of the forwarded request comprising the fingerprint value associated with the modified header of the request.

13. The apparatus of claim 12 , wherein the processor is further operative to:

determine whether the fingerprint is associated with a cached copy of the requested content object; and

in response to determining that the fingerprint is associated with the cached copy of the requested content object, respond to the request with the cached copy of the content object.

14. The apparatus of claim 12 , wherein the processor is further operative to:

determine whether the fingerprint is associated with a cached copy of the requested content object; and

in response to determining that the fingerprint is not associated with the cached copy of the requested content object:

remove the accept-encoding header from the request packet; and

re-forward the request toward a content provider associated with the content object.

15. A non-transitory computer-readable media comprising a software executed to:

receive a request packet comprising a request for a content object at a first proxy caching (PC) engine;

determine whether a cached copy associated with the requested content object exists in a content manager database; and

in response to determining that the cached copy of the content object does not exist in the content manager database:

modify the request packet to add an accept-encoding header to the request packet, the accept-encoding header providing an indication to an upstream PC engine that the first PC engine accepts a fingerprint associated with the content object,

forward the modified request packet toward a content provider associated with the requested content object, and

receive a response to the forwarded request packet from a second PC engine, the response comprising the forwarded request packet with a second modification in the accept-encoding header.

16. The software of claim 15 , further executed to:

determine whether the response comprises the fingerprint value associated with the accept-encoding header; and

in response to determining that the response comprises the fingerprint value associated with the accept-encoding header:

determine whether the fingerprint is associated the requested content object, and

in response to determining that the fingerprint is not associated with the requested content object:

remove the accept-encoding header from the request; and

re-forward the request toward the content provider associated with the requested content-object.

17. The software of claim 16 , further executed to:

in response to determining that the fingerprint is associated with the requested content object, retrieve the requested content object from the second PC engine.

18. The software of claim 16 , further executed to:

receive the requested content object from the content provider,

calculate a popularity for the requested content object,

determine whether the popularity of the requested content object exceeds a threshold value, and

in response to determining that the popularity of the requested content object exceeds a threshold value, cache a copy of the requested content object received from the content provider.

19. The software of claim 18 , further executed to:

receive a second data packet comprising a second request for the requested content object comprising a second accept-encoding header;

calculate a fingerprint for the cached copy of the requested content object;

modify a value of the second accept-encoding header to the calculated fingerprint; and

respond to the second request with the second data packet comprising the modified second accept-encoding header.