Cache purging in a distributed networked system
Purging resources from a cache in a distributed networked system is described. A first data center of the distributed networked system receives a purge request to purge a resource from cache. If the purge request does not include a cache key, the first data center determines whether the purge request is valid, and if valid, purges the resource from cache of the first data center, generates a cache key for the resource, and causes the purge request that includes the generated cache key to be sent to other data centers of the distributed networked system for purging the resource from cache. If the purge request includes a cache key, the first data center skips determining whether the purge request is valid and purges the resource from cache based on the cache key.
1. A method for purging resources from a cache in a distributed networked system, the method comprising:
receiving, at a first data center of a plurality of data centers of the distributed networked system, a first purge request to purge a first resource from cache;
determining that the first purge request does not include a first cache key for the first resource, and responsive to this determination:
determining that the first purge request is valid,
purging the first resource from cache of the first data center,
generating the first cache key for the first resource, and
causing a second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers of the distributed network system, the second purge request to purge the first resource;
receiving, at a second data center of the plurality of data centers of the distributed network system, the second purge request that includes the generated first cache key; and
determining that the second purge request includes the generated first cache key for the first resource, and responsive to this determination:
skipping determining whether the second purge request is valid, and
purging, based on the generated first cache key, the first resource from cache of the second data center.
2. The method of claim 1 , wherein the determining that the first purge request is valid includes:
determining that a requester of the first purge request is authorized to issue the first purge request for purging the first resource from cache; and
determining, based on one or more rules for the first resource, that the first resource can be stored in cache.
3. The method of claim 2 , wherein the one or more rules includes determining that a bypass cache setting is not applicable for the first resource.
4. The method of claim 1 , further comprising:
receiving, at the first data center, a third purge request to purge a second resource from cache; and
determining that the third purge request does not include a second cache key for the second resource, and responsive to this determination:
determining that the second purge request is not valid, and responsive to this determination, transmitting a message indicating that the second purge request cannot be performed.
5. The method of claim 1 , wherein the generating the first cache key for the first resource includes:
hashing a concatenated string of elements associated with the first resource to obtain the first cache key.
6. The method of claim 1 , wherein causing the second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers includes:
transmitting the second purge request to a purge queue manager that handles broadcasting the second purge request to the one or more other data centers of the plurality of data centers and managing a queue of purge requests.
7. The method of claim 6 , wherein the purge queue manager broadcasts the second purge request to each other data center of the plurality of data centers that are in a same region as the first data center and broadcasts the second purge request to a purge request distributor in each other region for broadcasting the second purge request to data centers in each other region.
8. A distributed networked system, comprising:
a set of one or more processors; and
a set of one or more non-transitory machine-readable storage mediums coupled with the set of one or more processors that stores instructions that, when executed by the set of one or more processors, causes the distributed networked system to perform operations including:
receiving, at a first data center of a plurality of data centers of the distributed networked system, a first purge request to purge a first resource from cache;
determining that the first purge request does not include a first cache key for the first resource, and responsive to this determination:
determining that the first purge request is valid,
purging the first resource from cache of the first data center,
generating the first cache key for the first resource, and
causing a second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers of the distributed network system, the second purge request to purge the first resource;
receiving, at a second data center of the plurality of data centers of the distributed network system, the second purge request that includes the generated first cache key; and
determining that the second purge request includes the generated first cache key for the first resource, and responsive to this determination:
skipping determining whether the second purge request is valid, and
purging, based on the generated first cache key, the first resource from cache of the second data center.
9. The distributed networked system of claim 8 , wherein the determining that the first purge request is valid includes:
determining that a requester of the first purge request is authorized to issue the first purge request for purging the first resource from cache; and
determining, based on one or more rules for the first resource, that the first resource can be stored in cache.
10. The distributed networked system of claim 9 , wherein the one or more rules includes determining that a bypass cache setting is not applicable for the first resource.
11. The distributed networked system of claim 8 , wherein the operations further comprise:
receiving, at the first data center, a third purge request to purge a second resource from cache; and
determining that the third purge request does not include a second cache key for the second resource, and responsive to this determination:
determining that the second purge request is not valid, and responsive to this determination, transmitting a message indicating that the second purge request cannot be performed.
12. The distributed networked system of claim 8 , wherein the generating the first cache key for the first resource includes:
hashing a concatenated string of elements associated with the first resource to obtain the first cache key.
13. The distributed networked system of claim 8 , wherein causing the second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers includes:
transmitting the second purge request to a purge queue manager that handles broadcasting the second purge request to the one or more other data centers of the plurality of data centers and managing a queue of purge requests.
14. The distributed networked system of claim 13 , wherein the purge queue manager broadcasts the second purge request to each other data center of the plurality of data centers that are in a same region as the first data center and broadcasts the second purge request to a purge request distributor in each other region for broadcasting the second purge request to data centers in each other region.
15. A non-transitory machine-readable storage medium that provides instructions that, if executed by one or more processors in a distributed network system will cause the distributed network system to perform operations comprising:
receiving, at a first data center of a plurality of data centers of the distributed networked system, a first purge request to purge a first resource from cache;
determining that the first purge request does not include a first cache key for the first resource, and responsive to this determination:
determining that the first purge request is valid,
purging the first resource from cache of the first data center,
generating the first cache key for the first resource, and
causing a second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers of the distributed network system, the second purge request to purge the first resource;
receiving, at a second data center of the plurality of data centers of the distributed network system, the second purge request that includes the generated first cache key; and
determining that the second purge request includes the generated first cache key for the first resource, and responsive to this determination:
skipping determining whether the second purge request is valid, and
purging, based on the generated first cache key, the first resource from cache of the second data center.
16. The non-transitory machine-readable storage medium of claim 15 , wherein the determining that the first purge request is valid includes:
determining that a requester of the first purge request is authorized to issue the first purge request for purging the first resource from cache; and
determining, based on one or more rules for the first resource, that the first resource can be stored in cache.
17. The non-transitory machine-readable storage medium of claim 16 , wherein the one or more rules includes determining that a bypass cache setting is not applicable for the first resource.
18. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise:
receiving, at the first data center, a third purge request to purge a second resource from cache; and
determining that the third purge request does not include a second cache key for the second resource, and responsive to this determination:
determining that the second purge request is not valid, and responsive to this determination, transmitting a message indicating that the second purge request cannot be performed.
19. The non-transitory machine-readable storage medium of claim 15 , wherein the generating the first cache key for the first resource includes:
hashing a concatenated string of elements associated with the first resource to obtain the first cache key.
20. The non-transitory machine-readable storage medium of claim 15 , wherein causing the second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers includes:
transmitting the second purge request to a purge queue manager that handles broadcasting the second purge request to the one or more other data centers of the plurality of data centers and managing a queue of purge requests.
21. The non-transitory machine-readable storage medium of claim 20 , wherein the purge queue manager broadcasts the second purge request to each other data center of the plurality of data centers that are in a same region as the first data center and broadcasts the second purge request to a purge request distributor in each other region for broadcasting the second purge request to data centers in each other region.