Intelligent caching for requests with query strings
An intelligent caching system is described herein that intelligently consolidates the name-value pairs in content requests containing query strings so that only substantially non-redundant responses are cached, thereby saving cache proxy resources. The intelligent caching system determines which name-value pairs in the query string can affect the redundancy of the content response and which name-value pairs can be ignored. The intelligent caching system organically builds the list of relevant name-value pairs by relying on a custom response header or other indication from the content server. Thus, the intelligent caching system results in fewer requests to the content server as well as fewer objects in the cache.
1. A computer-implemented method for normalizing a query string of a content request to reduce objects in a cache, the method comprising:
receiving a content identifier associated with a content item request, wherein the content identifier includes a query string with one or more name-value pairs;
determining whether an entry related to the received content identifier is present in a query string normalization table;
upon identifying a matching entry in the query string normalization table,
identifying irrelevant query name-value pairs in the received content identifier, based on the identified table entry, wherein a query name-value pair is irrelevant when removing it from the content identifier retrieves the same content item from the cache,
rewriting the content identifier to exclude identified irrelevant name-value pairs in the query string,
looking up the content item in the cache using the rewritten content identifier as a cache key;
determining that the content item is not present in the cache; and
forwarding the content item request to a second cache when the content item is determined not to be present in the cache wherein the content identifier is forwarded to the second cache in an un-rewritten format;
wherein the preceding steps are performed by at least one processor.
2. The method of claim 1 wherein receiving the content identifier comprises receiving a uniform resource locator (URL) in a Hypertext Transfer Protocol (HTTP) request from a client received by a cache server.
3. The method of claim 1 wherein the query normalization table includes information that identifies a uniform resource locator (URL) and one or more query string name-value pairs that are relevant for caching.
4. The method of claim 1 wherein the query normalization table includes information that identifies a uniform resource locator (URL) and one or more query string name-value pairs that are not relevant for caching.
5. The method of claim 1 wherein the query normalization table includes information that identifies a uniform resource locator (URL) and whether ordering of query string name-value pairs is relevant for caching.
6. The method of claim 1 wherein determining whether an entry is present comprises matching a key in the query normalization table with the content identifier having the query string removed.
7. The method of claim 1 further comprising matching the rewritten content identifier to a content item in a cache.
8. A computer-readable storage medium comprising instructions for controlling a computer system to generate a cache key for storing a content item, wherein the instructions, when executed, cause a processor to perform actions comprising:
receiving a content item identifier associated with a content item retrieved from a content server;
receiving query relevance information provided by the content server that indicates one or more elements of the content item identifier that are not relevant for caching, the query relevance information including a plurality of query name value pairs, wherein a query name-value pair is irrelevant when removing it from the content item identifier retrieves the same content item from a cache;
creating a cache key based on the received content item identifier by excluding at least one element of the content item identifier based on the received query relevance information, wherein the excluded at least one element does not affect a content response from the cache;
determining if the created cache key is already associated with an existing cache entry;
not storing the created cache key and the retrieved content item when the created cache key is determined to be associated with the existing cache entry; and
storing a cache entry containing the created cache key and the retrieved content item when the created cache key is determined not to be associated with the existing cache entry.
9. The medium of claim 8 wherein receiving the content item identifier comprises receiving a uniform resource locator (URL) that contains a query string with name-value pairs.
10. The medium of claim 8 wherein receiving query relevance information comprises receiving a custom Hypertext Transfer Protocol (HTTP) response header from the content server.
11. The medium of claim 8 further comprising updating a query normalization data structure based on the received query relevance information so that the system can normalize subsequent requests to exclude request elements not relevant for caching.
12. The medium of claim 8 wherein receiving query relevance information comprises receiving an expiration indication of the query relevance information indicating a period after which the query relevance information can be discarded.
13. The method of claim 1 wherein identifying irrelevant query name-value pairs in the received content identifier further comprises:
receiving from a requester information indicating one or more query names-value pairs are irrelevant.
14. The method of claim 1 wherein identifying irrelevant query name-value pairs in the received content identifier further comprises:
receiving from a requestor information indicating that one or more of query name-value pairs are relevant.