Dynamic expansion and contraction of edge clusters for managing access to cloud-based applications
Edge clusters execute in a plurality of regional clouds of a cloud computing platforms, which may include cloud POPs. Edge clusters may be programmed to control access to applications executing in the cloud computing platform. Edge clusters and an intelligent routing module route traffic to applications executing in the cloud computing platform. Cost and latency may be managed by the intelligent routing module by routing requests over the Internet or a cloud backbone network and using or bypassing cloud POPs. The placement of edge clusters may be selected according to measured or estimated latency. Latency may be estimated using speed test servers and the locations of speed test servers may be verified.
1 . A method comprising:
providing an application instance executing on a cloud computing platform including a plurality of regional clouds each associated with a geographic region and connected to one another by a cloud backbone network and by a wide area network WAN that is external to the cloud computing platform and does not include the cloud backbone network, the application instance executing in either a first regional cloud of the plurality of regional clouds or a computing device connected to the first regional cloud;
providing a first plurality of edge clusters having a first configuration including a first number of the first plurality of edge clusters and first locations of the first plurality of edge clusters in the plurality of regional clouds;
processing, by the first plurality of edge clusters, requests to access the application instance from a plurality of user endpoints in the WAN;
obtaining, by a computer module, a plurality of L1 values corresponding to latency between the plurality of user endpoints and the plurality of regional clouds;
obtaining, by the computer module, a plurality of L2 values corresponding to latency between pairs of regional clouds of the plurality of regional clouds; and
selecting, by the computer module, a second configuration of a second plurality of edge clusters according to the plurality of L1 values, the plurality of L2 values, and quantities of the requests received from across the geographic regions associated with the plurality of regional clouds, the second configuration including a second number of the second plurality of edge clusters and second locations of the first plurality of edge clusters, wherein one or both of (a) the second number is different from the first number and (b) the second locations are not all identical to the first locations;
wherein selecting the second configuration comprises executing an optimization algorithm with respect to the plurality of L1 values, the plurality of L2 values, and the quantities of the requests received from the geographic regions associated with the plurality of regional clouds;
wherein executing the optimization algorithm comprises executing the optimization algorithm subject to bounds to obtain the second configuration; and
wherein the bounds include at least one of a maximum increase of the second number relative to the first number and a maximum decrease of the second number relative to the first number.
2 . The method of claim 1 , further comprising outputting, by the computer module, a recommendation to implement the second configuration.
3 . The method of claim 1 , further comprising executing the optimization algorithm by evaluating each configuration of a plurality of configurations according to a cost function that is a function of the plurality of L1 values, the plurality of L2 values, the quantities of the requests received from the geographic regions associated with the plurality of regional clouds, and monetary costs for network usage and computational usage associated with each configuration.
4 . The method of claim 1 , wherein the bounds include a maximum permitted increase in latency experienced by the plurality of user endpoints relative to the first configuration.
5 . The method of claim 1 , further comprising characterizing cacheability of responses provided by the application instance in response to the requests and scaling down the plurality of L2 values according to the cacheability such that an amount of scaling down increases with increase in the cacheability.
6 . The method of claim 1 , further comprising scaling each L1 value according to one or both of a quantity of the requests received and sizes of the requests received from a geographic region corresponding to each L1 value.
7 . The method of claim 1 , further comprising:
evaluating the second configuration according to one or more validation criteria; and
determining that the second configuration satisfies the one or more validation criteria; and
in response to determining that the second configuration satisfies the one or more validation criteria, one or both of (a) outputting a report of the second configuration and (b) implementing the second configuration.
8 . The method of claim 7 , wherein the one or more validation criteria are a minimum improvement in overall latency of the second configuration relative to the first configuration.
9 . A system comprising:
a cloud computing platform comprising a plurality of regional clouds connected by a cloud backbone network, each regional cloud comprising a plurality of computing devices associated with a geographic region and connected by a regional network, the plurality of regional clouds being further connected to a wide area network (WAN) that does not include the cloud backbone network, an application instance executing in a first regional cloud of the plurality of regional clouds;
a first plurality of edge clusters having a first configuration including a first number of the first plurality of edge clusters and first locations of the first plurality of edge clusters in the plurality of regional clouds, the first plurality of edge clusters being configured to process requests to access the application instance from a plurality of user endpoints in the WAN; and
a computer module programmed to:
obtain a plurality of L1 values corresponding to latency between the plurality of user endpoints and the plurality of regional clouds;
obtain a plurality of L2 values corresponding to latency between pairs of regional clouds of the plurality of regional clouds; and
select a second configuration of a second plurality of edge clusters according to the plurality of L1 values, the plurality of L2 values, and quantities of the requests received from the geographic regions associated with the plurality of regional clouds, the second configuration including a second number of the second plurality of edge clusters and second locations of the first plurality of edge clusters, wherein one or both of (a) the second number is different from the first number and (b) the second locations are not all identical to the first locations;
wherein the computer module is further programmed to select the second configuration by executing an optimization algorithm with respect to the plurality of L1 values, the plurality of L2 values, and the quantities of the requests received from the geographic regions associated with the plurality of regional clouds; and
wherein the computer module is further programmed to execute the optimization algorithm by executing the optimization algorithm subject to a maximum increase of the second number relative to the first number and a maximum decrease of the second number relative to the first number.
10 . The system of claim 9 , wherein the computer module is further programmed to execute the optimization algorithm by evaluating each configuration of a plurality of configurations according to a cost function that is a function of the plurality of L1 values, the plurality of L2 values, the quantities of the requests received from the geographic regions associated with the plurality of regional clouds, and monetary costs for network usage and computational usage associated with each configuration.
11 . The system of claim 9 , wherein the computer module is further programmed to execute the optimization algorithm by executing the optimization algorithm subject to a maximum permitted increase in latency experienced by the plurality of user endpoints relative to the first configuration.
12 . The system of claim 9 , wherein the computer module is further programmed to characterize cacheability of responses provided by the application instance in response to the requests and scale down the plurality of L2 values according to the cacheability such that an amount of scaling down increases with increase in the cacheability.
13 . The system of claim 9 , wherein the computer module is further programmed to scale each L1 value according to a quantity of the requests received from a geographic region corresponding to each L1 value.
14 . The system of claim 9 , wherein the computer module is further programmed to:
evaluate the second configuration according to one or more validation criteria; and
if the second configuration satisfies the one or more validation criteria, one or both of (a) output a report of the second configuration and (b) implement the second configuration;
wherein the one or more validation criteria are a minimum improvement in overall latency of the second configuration relative to the first configuration.