Global capacity scaling
Dynamic scaling of a service in a CDN is performed using content request response times from service instances executing in regions in the CDN. Response times for similar content requests can be aggregated for each region. A cost calculation for each region may be evaluated in conjunction with the associated response time by the service instantiations to decide whether the service should be scaled up or down.
1. A method, comprising:
sending, by an edge gateway in a content delivery network, a content request from a user device to a region among one or more regions in the content delivery network where instances of a service are executing;
receiving, from an instance of a service executing in the region, a processing value that indicates a time from when the instance of the service started to receive the content request to when the instance of the service began to process to the request;
aggregating the processing value with a previous aggregated processing value associated with the region and one or more content requests that are similar to the content request;
calculating a service latency value for each region of the one or more regions using the associated aggregated processing value along with a round-trip time, wherein the round-trip time is measured between the edge gateway and that region;
calculating a cost of scaling up the service;
determining that the cost of scaling up the service is below a first threshold value and a service latency value for the service is below a second threshold value; and
based on the determination, scaling up the service in the content delivery network.
2. The method as recited in claim 1 , further comprising:
determining that the service latency value for the service exceeds a threshold value;
based on the determination that the service latency value for the service exceeds a threshold value, scaling down the service in the content delivery network.
3. The method as recited in claim 1 , further comprising:
periodically measuring the round-trip time between the edge gateway and each region of the one or more regions.
4. The method as recited in claim 1 , further comprising:
creating a hash key from the content request;
using the hash key to reference the previous aggregated processing value associated with the region.
5. One or more non-transitory computer-readable storage media, storing one or more sequences of instructions, which when executed by one or more processors cause performance of:
sending, by an edge gateway in a content delivery network, a content request from a user device to a region among one or more regions in the content delivery network where instances of a service are executing;
receiving, from an instance of a service executing in the region, a processing value that indicates a time from when the instance of the service started to receive the content request to when the instance of the service began to process to the request;
aggregating the processing value with a previous aggregated processing value associated with the region and one or more content requests that are similar to the content request;
calculating a service latency value for each region of the one or more regions using the associated aggregated processing value along with a round-trip time, wherein the round-trip time is measured between the edge gateway and that region;
calculating a cost of scaling up the service;
determining that the cost of scaling up the service is below a first threshold value and a service latency value for the service is below a second threshold value; and
based on the determination, scaling up the service in the content delivery network.
6. The one or more non-transitory computer-readable storage media as recited in claim 5 , further comprising:
determining that the service latency value for the service exceeds a threshold value;
based on the determination that the service latency value for the service exceeds a threshold value, scaling down the service in the content delivery network.
7. The one or more non-transitory computer-readable storage media as recited in claim 5 , further comprising:
periodically measuring the round-trip time between the edge gateway and each region of the one or more regions.
8. The one or more non-transitory computer-readable storage media as recited in claim 5 , further comprising:
creating a hash key from the content request;
using the hash key to reference the previous aggregated processing value associated with the region.
9. A system, comprising:
an edge gateway in a content delivery network, implemented at least partially in hardware, configured to send a content request from a user device to a region among one or more regions in the content delivery network where instances of a service are executing;
wherein the edge gateway is configured to receive, from an instance of a service executing in the region, a processing value that indicates a time from when the instance of the service started to receive the content request to when the instance of the service began to process to the request;
wherein the edge gateway is configured to aggregate the processing value with a previous aggregated processing value associated with the region and one or more content requests that are similar to the content request;
wherein the edge gateway is configured to calculate a service latency value for each region of the one or more regions using the associated aggregated processing value along with a round-trip time, wherein the round-trip time is measured between the edge gateway and that region;
a service scaler, implemented at least partially in hardware, configured to calculate a cost of scaling up the service, determine that the cost of scaling up the service is below a first threshold value and the service latency value for the service is below a second threshold value.
10. The system as recited in claim 9 , wherein the service scaler is configured to:
determine that the service latency value for the service exceeds a threshold value;
based on the determination that the service latency value for the service exceeds a threshold value, scale down the service in the content delivery network.
11. The system as recited in claim 9 , wherein the edge gateway is configured to periodically measure the round-trip time between the edge gateway and each region of the one or more regions.
12. The system as recited in claim 9 , wherein the edge gateway is configured to:
create a hash key from the content request;
use the hash key to reference the previous aggregated processing value associated with the region.