End-to-end content delivery network incorporating independently operated transparent caches and proxy caches
Some embodiments provide an end-to-end federated CDN solution that assimilates a transparent caching server that is operated by a transparent caching server operator into a CDN that independently operates CDN caching servers. Specifically, the logs from the transparent caching server are assimilated into the CDN by aggregating the logs from the transparent caching server and processing the transparent caching server logs to identify network usage for content of a CDN content provider customer that is delivered by the transparent caching server. The network usage is then combined with the network usage that tracked by the CDN caching servers in order to provide comprehensive report metrics for the content provider customer and to bill the content provider customer for all network usage related to delivering the content provider customer's content irrespective of whether the content was delivered by a transparent caching server or a CDN caching server.
1. A method comprising:
providing a proxy server in a network path between a source and at least first and second destinations, wherein the proxy server is assigned a first address identifier that is different than a second address identifier assigned to the source, and wherein the proxy server operates independent of said source;
receiving at the proxy server, a plurality of content passing from the source over the network path to the first destination;
caching by said proxy server, first content of the plurality of content that includes a particular key introduced into the first content by the source;
forwarding, without caching at the proxy server, second content of the plurality of content that does not include the particular key;
intercepting at the proxy server, a first request from the second destination, the first request comprising a destination address set to the second address identifier of said source and the particular key;
responding to the first request with the proxy server providing the second destination with a response comprising said first content and a source address, and wherein said responding comprises inserting the second address identifier, instead of the first address identifier of the proxy server, as the source address of the response from said proxy server;
tracking at the proxy server, latency between the proxy server and the second destination while responding to the first request;
adding a particular entry to a plurality of entries in a log of the proxy server in response to said responding and said tracking, wherein adding the particular entry comprises entering as part of the particular entry (i) the latency, (ii) an identifier of the first content, (iii) an amount of the first content provided by the proxy server to the second destination, and (iv) the first address identifier of the proxy identifier, instead of the second address identifier, as the source address of the particular entry, and wherein each entry of the plurality of entries stores latency, a content identifier, a distributed amount of content, and the first address identifier in association with the proxy server responding to a different request comprising a destination address set to the second address identifier of the source; and
providing the log from the proxy server to an entity with control over the source.
2. The method of claim 1 further comprising computing overall latency resulting from delivery of said first content based on said latency tracked by the proxy server and latency tracked from separate delivery of said first content from said source.
3. The method of claim 1 , wherein intercepting said first request comprises detecting the particular key in said first request.
4. The method of claim 3 , wherein said particular key comprises at least one of a particular domain name, a Uniform Resource Locator (URL), or header field included with said first request.
5. The method of claim 3 further comprising receiving a second request at the proxy server, and passing the second request through the proxy server in response to the second request omitting said particular key, wherein said passing comprises receiving and distributing the second request from the proxy server to an intended recipient, and wherein said intercepting comprises receiving the first request and preventing distribution of the first request past the proxy server to the source or another intended recipient.
6. The method of claim 1 , wherein said tracking comprises recording bandwidth consumed in distributing said first content from the proxy server to the second destination.
7. The method of claim 1 further comprising intercepting a second request, that is directed to a third address identifier associated with a different second source, and responding to the second request by providing third content of the second source from said proxy server with a source address set to the third address identifier associated with said second source instead of the first address identifier of the proxy server.
8. The method of claim 7 further comprising tracking second latency that results from responding to the second request, and entering the second latency to a different entry, than the particular entry, in said log.
9. The method of claim 1 , wherein the latency between the proxy server and the second destination is less than the latency between said source and said second destination.
10. The method of claim 1 further comprising integrating said proxy server as a content distribution point of the entity controlling said source by incorporating the latency between the proxy server and the second destination with latency resulting from serving the first content from the said source.
11. The method of claim 1 further comprising providing said proxy server with a plurality of keys, the plurality of keys identifying said first content and other content designated for caching and distribution from said proxy server, and wherein the plurality of keys do not include a key for the second content.
12. The method of claim 11 further comprising reconfiguring said proxy server based on said plurality of keys, wherein said reconfiguring comprises performing a prioritized caching of content identified by the plurality of keys, and performing a deprioritized caching of content not identified by the plurality of keys.