Peer cache filling
The embodiments provide peer cache filling. The peer cache filling allocates a set of caching servers to distribute content in response to user requests with a limited first subset of the set of servers having access to retrieve the content from an origin and with a larger second subset of the set of servers retrieving the content from the first subset of servers without accessing the origin. The peer cache filling dynamically escalates and deesclataes the allocation of the caching servers to the first and second subsets as demand for the content rises and falls. Peer cache filling is implemented by modifying request headers to identify designated hot content, provide a request identifier hash result for identifying the ordering of servers, and provide a value for designating which servers in the ordering as primary server with access to the origin.
1. A method comprising:
tracking a request rate for particular content;
scaling from a single first server responding to user requests for the particular content to a first set of servers responding to the user requests for the particular content based on the request rate satisfying a first threshold, wherein the first set of servers comprises first, second, and third servers;
providing, based on a size of the first set of servers and in response to the request rate satisfying the first threshold, the first server with access to an origin server storing the particular content, and the second and third servers with access to retrieve the particular content from the first server;
filling caches of the first set of servers with the particular content by said first server retrieving and locally storing a copy of the particular content from the origin server, and by said second and third servers directly requesting and retrieving said copy of the particular content from the first server without accessing the origin server or other device;
scaling from the first set of servers, responding to the user requests for the particular content, to a larger second set of servers responding to the user requests for the particular content based on the request rate satisfying a different second threshold, wherein the second set of servers comprises the first, second, and third servers from the first set of servers and a fourth server not in the first set of servers;
providing, based on a size of the second set of servers and in response to the request rate satisfying the second threshold, the first server and the second server with access to the origin server, and the third server and the fourth server with access to retrieve the particular content from the first server or the second server;
filling caches of the second set of servers with the particular content by said first and second servers retrieving and locally storing the copy of the particular content from the origin server, and by said third and fourth servers directly requesting and retrieving said copy of the particular content from one of the first and second servers without accessing the origin server or other device; and
serving the copy of the particular content from the local storage of each server of the second set of servers over a data network to a plurality of users in response to the user requests being distributed across the second set of servers as a result of the request rate satisfying the second threshold.
2. The method of claim 1 further comprising distributing each of a first set of requests exclusively to the first server in response to the request rate immediately preceding the first set of requests not satisfying the first threshold.
3. The method of claim 2 further comprising distributing a second set of requests across the first set of servers in response to the request rate immediately preceding the second set of requests satisfying the first threshold without satisfying the second threshold.
4. The method of claim 3 further comprising distributing a third set of requests across the second set of servers in response to the request rate immediately preceding the third set of requests satisfying the second threshold.
5. The method of claim 3 , wherein distributing the first set of requests comprises distributing the first set of requests without modification, and wherein distributing the second set of requests comprises modifying a header of the second set of requests with a value designating the particular content as hot content.
6. The method of claim 1 , wherein scaling from the single first server comprises modifying a header of said user requests with a first value designating the first server as an exclusive primary server from which the second and third servers retrieve copies of the particular content, said first value further prohibiting the second and third servers from accessing the origin server.
7. The method of claim 6 , wherein scaling from the first set of servers comprises modifying a header of said user requests with a different second value designating the first server and the second server as primary servers from which the third and fourth servers retrieve copies of the particular content, said second value further prohibiting the third and fourth servers from accessing the origin server.
8. A method comprising: tracking a rate of requests for particular content received at a point-of-presence (PoP), the PoP comprising a plurality of servers operating collectively to distribute different content provider content;
receiving a first request for the particular content at a first server of the plurality of servers;
modifying a cache fill operation of the first server based on the rate of requests, wherein modifying the cache fill operation of the first server comprises:
retrieving to the first server, the particular content from a second server of the plurality of servers in response to the rate of requests surpassing a first threshold;
retrieving to the first server, the particular content directly from an origin server originating the particular content in response to the rate of requests surpassing a different second threshold, wherein the origin server is located outside of the PoP;
serving the particular content from the first server in response to the first request;
receiving a second request for the particular content at a third server of the plurality of servers; and
modifying a cache fill operation of the third server based on the rate of requests, wherein modifying the cache fill operation of the third server comprises:
retrieving to the third server, the particular content from the second server in response to the rate of requests surpassing the first threshold; and
retrieving to the third server, the particular content from one of the second server and the first server in response to the rate of requests surpassing the second threshold.
9. The method of claim 8 further comprising distributing a set of requests across the first server and the second server in response to the rate of requests surpassing the first threshold, and distributing the set of requests across the first server, the second server, and the third server in response to the rate of requests surpassing the second threshold.
10. The method of claim 8 , wherein the first threshold designates the particular content to a first hotness level, the first hotness level involving distribution from a first set of the plurality of servers, and the first set of servers comprising the first and second servers with the second server having exclusive access to the origin server, and wherein the second threshold designates the particular content to a second hotness level, the second hotness level involving distribution from a different second set of the plurality of servers, and the second set of servers comprising the first server, the second server, and the third server with the first and second servers having access to the origin server.
11. The method of claim 8 further comprising hashing a Uniform Request Identifier (URI) of said first request, and selecting the first server to receive said request based on a result of said hashing.
12. The method of claim 11 further comprising inserting the result of said hashing in a header of the first request in response to the rate of requests surpassing the first threshold.
13. The method of claim 11 further comprising ordering the plurality of servers based on the result of said hashing.
14. The method of claim 13 further comprising selecting the second server as a primary server from which the first server retrieves the copy of the particular content based on said ordering.
15. The method of claim 11 further comprising inserting in a header of the first request, a value designating the second server as a primary server from which the first server retrieves the copy of the particular content.
16. A distributed platform comprising:
at least one point-of-presence (PoP) comprising:
a front-end server, by operation of a processor and a network interface, distributing requests for particular content across a plurality of back-end servers; and
the plurality of back-end servers serving the particular content in response to said requests, the plurality of back-end servers comprising:
(i) a first back-end server, by operation of a processor, a memory providing a cache, and a network interface, retrieving the particular content directly from an origin server that stores an original copy of the particular content and that is located outside the PoP,
(ii) a second back-end server, by operation of a processor, a memory providing a cache, and a network interface, exclusively retrieving the particular content from the first back-end server without access to the origin server in response to a first rate of requests for the particular content and a request with a first modified header comprising a first identifier identifying the first back-end server, and retrieving the particular content directly from the origin server in response to a second rate of requests that is greater than the first rate of requests, and
(iii) a third back-end server, by operation of a processor, a memory providing a cache, and a network interface, exclusively retrieving the particular content from the first back-end server without access to the origin server in response to the first rate of requests and a request with the first modified header comprising the first identifier identifying the first back-end server, and retrieving the particular content from one of the first back-end server or the second back-end server without access to the origin server in response to the second rate of requests and a request with a second modified header comprising the first identifier and a second identifier identifying the second back-end server.
17. The distributed platform of claim 16 , wherein the front-end server distributing the requests comprises distributing the requests exclusively to the first back-end server in response to a rate of requests being less than the first rate of requests, distributing the requests across the first and second back-end servers in response to the first rate of requests, and distributing the requests across the first, second, and third back-end servers in response to the second rate of requests.
18. The distributed platform of claim 16 , wherein the front-end server comprises memory tracking request counts for the particular content.