Efficient cache validation and content retrieval in a content delivery network
Some embodiments provide systems and methods for validating cached content based on changes in the content instead of an expiration interval. One method involves caching content and a first checksum in response to a first request for that content. The caching produces a cached instance of the content representative of a form of the content at the time of caching. The first checksum identifies the cached instance. In response to receiving a second request for the content, the method submits a request for a second checksum representing a current instance of the content and a request for the current instance. Upon receiving the second checksum, the method serves the cached instance of the content when the first checksum matches the second checksum and serves the current instance of the content upon completion of the transfer of the current instance when the first checksum does not match the second checksum.
1. A computer-implemented method comprising:
caching, responsive to a first request for specific content, the specific content and a first checksum, wherein caching the specific content produces a cached instance of the specific content representative of a form of the specific content at the time of caching, and wherein the first checksum identifies the cached instance of the specific content;
receiving a second request for the specific content;
submitting a request for a second checksum, the second checksum identifying a current instance of the specific content, and wherein the current instance is representative of a present form of the specific content;
submitting a request for the current instance of the specific content;
receiving the second checksum identifying the current instance of the specific content contemporaneous with transfer of the current instance of the specific content commencing;
serving, responsive to the second request, the cached instance of the specific content without waiting for the transfer of the current instance to complete when the first checksum matches the second checksum; and
serving, responsive to the second request, the current instance of the specific content upon completion of the transfer of the current instance when the first checksum does not match the second checksum.
2. The computer-implemented method of claim 1 further comprising replacing the cached instance of the specific content with the current instance when the first checksum does not match the second checksum.
3. The computer-implemented method of claim 1 further comprising storing an expiration parameter when caching the specific content.
4. The computer-implemented method of claim 3 further comprising serving, in response to the second request, the cached instance of the specific content when the expiration parameter has not expired.
5. The computer-implemented method of claim 3 further comprising refreshing the expiration parameter based on an expiration parameter of the current instance of the specific content when the first checksum matches the second checksum.
6. The computer-implemented method of claim 1 further comprising computing the first checksum based on a hash of an identifier identifying the specific content.
7. The computer-implemented method of claim 1 further comprising terminating the transfer of the current instance of the specific content when the first checksum does not match the second checksum.
8. The computer-implemented method of claim 1 , wherein the first checksum is a numeric value representing the form of the specific content at the time of caching and wherein the second checksum is a numeric value representing the present form of the specific content.
9. The computer-implemented method of claim 1 , wherein the request for the second checksum and the request for the current instance of the specific content are submitted as a single datagram.
10. The computer-implemented method of claim 1 , wherein the request for the second checksum and the request for the current instance of the specific content are submitted as separate datagrams.
11. A computer-implemented method performed by a first server in a first cache tier of a cache hierarchy to support cache validation function of a second server in a second cache tier of the cache hierarchy, the computer implemented method comprising:
maintaining a current instance of specific content in cache of the first server by refreshing the current instance of the specific content when a time-to-live parameter associated with the specific content expires;
computing a first checksum upon refresh of the current instance, the first checksum identifying a present form of the specific content as represented by the current instance of the specific content;
receiving, from the second server, a request for the first checksum and the current instance of the specific content;
passing the checksum to the second server contemporaneously with commencing transfer of the current instance of the specific content to the second server;
completing transfer of the current instance of the specific content to invalidate a previously cached instance of the specific content stored to a cache of the second server when the second server identifies that the first checksum differs from a second checksum, wherein the second checksum identifies a form of the specific content represented by the cached instance stored to the cache of the second server; and
interrupting transfer of the current instance of the specific content to prevent invalidation of the previously cached instance of the specific content when the second server identifies that the first checksum does not differ from the second checksum.
12. The computer-implemented method of claim 11 further comprising receiving a request to interrupt transfer of the current instance when the first checksum does not differ from the second checksum.
13. The computer-implemented method of claim 11 , wherein the first server and the second server are servers of a content delivery network.
14. The computer-implemented method of claim 11 , wherein the request for the first checksum and the current instance of the specific content comprises a uniform resource locator (URL) for the specific content.
15. The computer-implemented method of claim 14 further comprising identifying the first checksum based on the URL of the request.
16. The computer-implemented method of claim 11 , wherein computing the first checksum comprises performing a hash of the current instance of the specific content to produce a numeric value representing the first checksum.
17. The computer-implemented method of claim 11 , wherein computing the first checksum comprises computing the first checksum on a first partition of a plurality of partitions that comprise the current instance of the specific content.
18. A distributed system comprising:
a first caching tier server comprising:
a first interface communicably coupling the first caching tier server to an origin server;
a storage maintaining a current instance of specific content originated by the origin server and a first checksum identifying the current instance;
a second caching tier server comprising:
a first interface communicably coupling the second caching tier server to the first caching tier server;
a second interface communicably coupling the second caching tier server to a set of end users;
a storage caching a cached instance of the specific content and a second checksum identifying the cached instance of the specific content, wherein the second caching tier server serves the cached instance of the specific content to a first end user of the set of end users in response to a first request for the specific content; and
a processor, in response to a second request for the specific content, serving the cached instance of the specific content to a second end user of the set of end users over the second interface when the first checksum matches the second checksum and serving the current instance of the specific content to the second end user over the second interface when the first checksum does not match the second checksum.
19. The distributed system of claim 18 , wherein the second caching tier server requests, in response to the second request, the first checksum and the current instance of the specific content from the first caching tier server over the first interface of second caching tier server.
20. The distributed system of claim 18 , wherein the first caching tier server further comprises a processor to compute the first checksum for the current instance of the specific content and the second checksum for the cached instance of the specific content.