Intelligent autoscale of services
Autoscaling comprises: accessing operational data associated with a pool of servers; determining an expected load associated with a service provided by the pool of servers based at least in part on the operational data; determining an estimated capacity associated with the service provided by the pool of servers; determining that an autoscale operation is to be performed based at least in part on the expected load and the estimated capacity, wherein the autoscale operation automatically scales out one or more additional servers or automatically scales in one or more servers in the pool of servers; and performing the autoscale operation.
1. A method of performing an autoscale operation to adjust a set of target-application servers that receive data message flows from a set of clients through a plurality of service engines, the service engines maintaining operational data regarding the data message flows forwarded by the service engines to the set of servers, the method comprising:
collecting, from the plurality of service engines, operational data associated with a set of target-application servers to which the plurality of service engines forwards data message flows;
calculating an expected load on the set of target-application servers based on the collected operational data;
calculating an expected operational capacity of the set of target-application servers based on capacity data received at the plurality of service engines in headers of data messages from the set of target-application servers processed by the plurality of service engines;
based at least in part on the calculated expected load and capacity, determining that an autoscale operation is to be performed to add one or more additional target-application servers to, or remove one or more target-application servers from, the set of target-application servers; and
performing the autoscale operation.
2. The method of claim 1 , wherein calculating the expected load comprises calculating a range of a plurality of load values and selecting one of the values in the range as the expected load.
3. The method of claim 2 , wherein selecting the value comprises selecting a highest value in the range.
4. The method of claim 3 , wherein calculating the range comprises calculating a variance of historical load data in the previously collected operational data and using the calculated variance and a predicted load value to define the range.
5. The method of claim 4 , wherein calculating the variance of historical load data comprises:
aggregating the operational data associated with the set of target-application servers;
determining a historical moving average based at least in part on the aggregated operational data;
constructing a distribution model based at least in part on the historical moving average; and
calculating the variance based at least in part on the distribution model,
wherein the predicted load value is a value selected from the constructed distributed model.
6. The method of claim 5 , wherein the value selected from the constructed distributed model is a most likely value in the distribution model.
7. The method of claim 2 , wherein the range is calculated based on Holt-Winters algorithm.
8. The method of claim 1 , wherein the expected operational capacity is calculated based at least in part on one or more of: number of open connections, average number of requests/second, average number of connections/second, bandwidth, latency, connection quality, errors, service level agreement (SLA) constraints, and Application Performance Index (Apdex).
9. The method of claim 1 , wherein the capacity data in the headers comprises at least one of queue depth, wait time before the thread processes a new incoming connection, wait time for the next thread in scheduler queue, CPU utilization, memory utilization, and IO wait queue.
10. The method of claim 1 , wherein the expected operational capacity is calculated based at least in part on a threshold associated with a capacity model.
11. The method of claim 10 , wherein the threshold is a threshold of a rate of change of a capacity estimation factor.
12. The method of claim 1 , wherein the headers of the responses from the set of target-application servers comprise special Hypertext Transfer Protocol (HTTP) headers.
13. The method of claim 1 , wherein calculating the expected operational capacity comprises:
clustering historical operational data into a plurality of categories based on performance metrics; and
establishing a capacity model for the plurality of categories based at least in part on the historical operational data.
14. A non-transitory machine readable medium storing a program for execution by a set of processing units, the program for performing an autoscale operation to adjust a set of target-application servers that receive data message flows from a set of clients through a plurality of service engines, the service engines maintaining operational data regarding the data message flows forwarded by the service engines to the set of servers, the program comprising a set of instructions for:
collecting, from the plurality of service engines, operational data associated with a set of target-application servers to which the plurality of service engines forwards data message flows;
using previously collected operational data to calculate a variance in a load value expressing the load on the set of target-application servers;
using the calculated variance to calculate an expected load on the set of target-application servers;
calculating an expected operational capacity of the set of target-application servers;
based at least in part on the calculated expected load and capacity, determining that an autoscale operation is to be performed to add one or more additional target-application servers to, or remove one or more target-application servers from, the set of target-application servers; and
performing the autoscale operation.
15. The non-transitory machine readable medium of claim 14 , wherein the set of instructions for using the calculated variance to calculate the expected load on the set of target-application servers comprises a set of instructions for calculating a range of a plurality of load values and selecting one of the values in the range as the expected load.
16. The non-transitory machine readable medium of claim 15 , wherein the set of instructions for using the calculated variance to calculate an expected load on the set of target-application servers further comprises sets of instructions for:
aggregating the operational data associated with the set of target-application servers;
determining a historical moving average based at least in part on the aggregated operational data;
constructing a distribution model based at least in part on the historical moving average;
calculating the variance based at least in part on the distribution model; and
using the calculated variance and a predicted load value to define the range,
wherein the predicted load value is a value selected from the constructed distributed model.
17. An autoscaling system for performing an autoscale operation to adjust a set of target-application servers that receive data message flows from a set of clients through a plurality of service engines, the service engines maintaining operational data regarding the data message flows forwarded by the service engines to the set of servers, comprising:
a non-transitory machine readable medium storing a program for execution by a set of processing units, the program comprising sets of instructions for:
collecting, from the plurality of service engines, operational data associated with a set of target-application servers to which the plurality of service engines forwards data message flows;
calculating an expected load on the set of target-application servers based on the collected operational data;
calculating an expected operational capacity of the set of target-application servers based on capacity data received at the plurality of service engines in headers of data messages from the set of target-application servers processed by the plurality of service engines;
based at least in part on the calculated expected load and capacity, determining that an autoscale operation is to be performed to add one or more additional target-application servers to, or remove one or more target-application servers from, the set of target-application servers; and
performing the autoscale operation.
18. The autoscaling system of claim 17 , wherein the headers of the responses from the set of target-application servers comprise special Hypertext Transfer Protocol (HTTP) headers.
19. The autoscaling system of claim 17 , wherein the capacity data in the headers comprises at least one of queue depth, wait time before the thread processes a new incoming connection, wait time for the next thread in scheduler queue, CPU utilization, memory utilization, and TO wait queue.
20. The non-transitory machine readable medium product of claim 16 , wherein the value selected from the constructed distributed model is a most likely value in the distribution model.