Cache purging in a distributed networked system
Purging resources from a cache in a distributed networked system is described. A compute server of a first data center of the distributed networked system receives a purge request to purge a resource from cache. If the purge request does not include a cache key, the compute server determines whether the purge request is valid, and if valid, purges the resource from cache of the first data center, generates a cache key for the resource, and causes the purge request that includes the generated cache key to be sent to other data centers of the distributed networked system for purging the resource from cache. If the purge request includes a cache key, the compute server skips determining whether the purge request is valid and purges the resource from cache based on the cache key.
1. A method for purging resources from a cache in a distributed networked system, the method comprising:
receiving, at a first data center of a plurality of data centers of the distributed networked system, a first purge request to purge a first resource from cache;
determining that the first purge request does not include a first cache key for the first resource, and responsive to this determination:
determining that the first purge request is valid,
purging the first resource from cache of the first data center,
generating the first cache key for the first resource, and
causing a second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers of the distributed network system, the second purge request to purge the first resource;
receiving, at the first data center of the plurality of data centers of the distributed network system, a third purge request to purge a second resource from cache; and
determining that the third purge request includes a second cache key for the second resource, and responsive to this determination:
not performing a determination of whether the third purge request is valid, and
purging, based on the second cache key, the second resource from cache of the first data center.
2. The method of claim 1 , wherein the determining that the first purge request is valid includes:
determining that a requester of the first purge request is authorized to issue the first purge request for purging the first resource from cache; and
determining, based on one or more rules for the first resource, that the first resource can be stored in cache.
3. The method of claim 2 , wherein the one or more rules includes determining that a bypass cache setting is not applicable for the first resource.
4. The method of claim 1 , further comprising:
receiving, at the first data center, a fourth purge request to purge a third resource from cache; and
determining that the fourth purge request does not include a third cache key for the third resource, and responsive to this determination:
determining that the fourth purge request is not valid, and responsive to this determination, transmitting a message indicating that the fourth purge request cannot be performed.
5. The method of claim 1 , wherein the generating the first cache key for the first resource includes:
hashing a concatenated string of elements associated with the first resource to obtain the first cache key.
6. The method of claim 1 , wherein causing the second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers includes:
transmitting the second purge request to a purge queue manager that handles broadcasting the second purge request to the one or more other data centers of the plurality of data centers and managing a queue of purge requests.
7. The method of claim 6 , wherein the purge queue manager broadcasts the second purge request to each other data center of the plurality of data centers that are in a same region as the first data center and broadcasts the second purge request to a purge request distributor in each other region for broadcasting the second purge request to data centers in each other region.
8. A non-transitory machine-readable storage medium that provides instructions that, if executed by one or more processors of a compute server of a distributed network system will cause the compute server to perform operations comprising:
receiving, at the compute server of the distributed networked system, a first purge request to purge a first resource from cache, wherein the compute server is part of a first data center of a plurality of data centers of the distributed networked system;
determining that the first purge request does not include a first cache key for the first resource, and responsive to this determination:
determining that the first purge request is valid,
purging the first resource from cache of the first data center,
generating the first cache key for the first resource, and
causing a second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers of the distributed network system, the second purge request to purge the first resource;
receiving, at the compute server, a third purge request to purge a second resource from cache; and
determining that the third purge request includes a second cache key for the second resource, and responsive to this determination:
not performing a determination of whether the third purge request is valid, and
purging, based on the second cache key, the second resource from cache of the first data center.
9. The non-transitory machine-readable storage medium of claim 8 , wherein the determining that the first purge request is valid includes:
determining that a requester of the first purge request is authorized to issue the first purge request for purging the first resource from cache; and
determining, based on one or more rules for the first resource, that the first resource can be stored in cache.
10. The non-transitory machine-readable storage medium of claim 9 , wherein the one or more rules includes determining that a bypass cache setting is not applicable for the first resource.
11. The non-transitory machine-readable storage medium of claim 8 , wherein the operations further comprise:
receiving, at the compute server, a fourth purge request to purge a third resource from cache; and
determining that the fourth purge request does not include a third cache key for the third resource, and responsive to this determination:
determining that the fourth purge request is not valid, and responsive to this determination, transmitting a message indicating that the fourth purge request cannot be performed.
12. The non-transitory machine-readable storage medium of claim 8 , wherein the generating the first cache key for the first resource includes:
hashing a concatenated string of elements associated with the first resource to obtain the first cache key.
13. The non-transitory machine-readable storage medium of claim 8 , wherein causing the second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers includes:
transmitting the second purge request to a purge queue manager that handles broadcasting the second purge request to the one or more other data centers of the plurality of data centers and managing a queue of purge requests.
14. The non-transitory machine-readable storage medium of claim 13 , wherein the purge queue manager broadcasts the second purge request to each other data center of the plurality of data centers that are in a same region as the first data center and broadcasts the second purge request to a purge request distributor in each other region for broadcasting the second purge request to data centers in each other region.
15. A compute server of a distributed networked system, comprising:
a set of one or more processors; and
a set of one or more non-transitory machine-readable storage mediums coupled with the set of one or more processors that stores instructions that, when executed by the set of one or more processors, causes the compute server of the distributed networked system to perform operations including:
receiving, at the compute server of the distributed network system, a first purge request to purge a first resource from cache, wherein the compute server is part of a first data center of a plurality of data centers of the distributed networked system;
determining that the first purge request does not include a first cache key for the first resource, and responsive to this determination:
determining that the first purge request is valid,
purging the first resource from cache of the first data center,
generating the first cache key for the first resource, and
causing a second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers of the distributed network system, the second purge request to purge the first resource;
receiving, at the compute server, a third purge request to purge a second resource from cache; and
determining that the third purge request includes a second cache key for the second resource, and responsive to this determination:
not performing a determination of whether the third purge request is valid, and
purging, based on the second cache key, the second resource from cache of the first data center.
16. The compute server of the distributed networked system of claim 15 , wherein the determining that the first purge request is valid includes:
determining that a requester of the first purge request is authorized to issue the first purge request for purging the first resource from cache; and
determining, based on one or more rules for the first resource, that the first resource can be stored in cache.
17. The compute server of the distributed networked system of claim 16 , wherein the one or more rules includes determining that a bypass cache setting is not applicable for the first resource.
18. The compute server of the distributed networked system of claim 15 , wherein the operations further comprise:
receiving, at the compute server, a fourth purge request to purge a third resource from cache; and
determining that the fourth purge request does not include a third cache key for the third resource, and responsive to this determination:
determining that the fourth purge request is not valid, and responsive to this determination, transmitting a message indicating that the fourth purge request cannot be performed.
19. The compute server of the distributed networked system of claim 15 , wherein the generating the first cache key for the first resource includes:
hashing a concatenated string of elements associated with the first resource to obtain the first cache key.
20. The compute server of the distributed networked system of claim 15 , wherein causing the second purge request that includes the generated first cache key to be sent to one or more other data centers of the plurality of data centers includes:
transmitting the second purge request to a purge queue manager that handles broadcasting the second purge request to the one or more other data centers of the plurality of data centers and managing a queue of purge requests.