Trajectory-based hierarchical autoscaling for serverless applications

1. A method comprising:

monitoring, during a first time interval, traffic associated with one or more applications executed by a cluster of compute nodes, wherein each compute node of the cluster of compute nodes comprises a virtual machine;

scaling a number of replicas of the one or more applications based on the traffic associated with the one or more applications over a second time interval, the second time interval corresponding to an amount of time associated with instantiating a replica of the one or more applications;

determining, by a processing device, in view of the traffic associated with the one or more applications during the first time interval, that the traffic is predicted to exceed a capacity threshold of the cluster of compute nodes at an end of a third time interval, wherein the third time interval is longer than the second time interval; and

initiating startup of an additional compute node comprising an additional virtual machine to be added to the cluster of compute nodes for executing replicas of the one or more applications based on the traffic being predicted to exceed the capacity threshold of the cluster of compute nodes at the end of the third time interval, wherein the third time interval corresponds to an amount of time associated with starting up the additional compute node.

2. The method of claim 1 , wherein the one or more applications comprise one or more serverless applications.

3. The method of claim 1 , wherein the traffic comprises a number of concurrent requests received by each of the one or more applications executed by the cluster of compute nodes.

4. The method of claim 1 , wherein monitoring the traffic associated with the one or more applications executed by the cluster of compute nodes comprises:

scraping one or more traffic metrics from each of the one or more applications; and

determining a total traffic level of the cluster during the first time interval in view of the one or more traffic metrics from each of the one or more applications.

5. The method of claim 4 , wherein determining that the traffic associated with the one or more applications is predicted to exceed the capacity threshold of the cluster of compute nodes at the end of the third time interval comprises:

extrapolating the total traffic level of the cluster during the first time interval over the third time interval.

6. The method of claim 5 , wherein extrapolating the total traffic level of the cluster comprises:

applying a statistical analysis or machine learning model on the total traffic level of the cluster during the first time interval to estimate a future total traffic level of the cluster at the end of the third time interval.

7. A system comprising:

a memory; and

a processing device, operatively coupled to the memory, the processing device to:

monitor, during a first time interval, traffic associated with one or more applications executed by a cluster of compute nodes, wherein each compute node of the cluster of compute nodes comprises a virtual machine;

scale a number of replicas of the one or more applications based on the traffic associated with the one or more applications over a second time interval, the second time interval corresponding to an amount of time associated with instantiating a replica of the one or more applications;

determine, in view of the traffic associated with the one or more applications during the first time interval, that the traffic is predicted to exceed a capacity threshold of the cluster of compute nodes at an end of a third time interval, wherein the third time interval is longer than the second time interval; and

initiate startup of an additional compute node comprising an additional virtual machine to be added to the cluster of compute nodes for executing replicas of the one or more applications based on the traffic being predicted to exceed the capacity threshold of the cluster of compute nodes at the end of the third time interval, wherein the third time interval corresponds to an amount of time associated with starting up the additional compute node.

8. The system of claim 7 , wherein the one or more applications comprise one or more serverless applications.

9. The system of claim 7 , wherein the traffic comprises a number of concurrent requests received by each of the one or more applications executed by the cluster of compute nodes.

10. The system of claim 7 , wherein to monitor the traffic associated with the one or more applications executed by the cluster of compute nodes, the processing device is to:

scrape one or more traffic metrics from each of the one or more applications; and

determine a total traffic level of the cluster during the first time interval in view of the one or more traffic metrics from each of the one or more applications.

11. The system of claim 10 , wherein to determine that the traffic associated with the one or more applications is predicted to exceed the capacity threshold of the cluster of compute nodes at the end of the third time interval, the processing device is to:

extrapolate the total traffic level of the cluster during the first time interval over the third time interval.

12. The system of claim 11 , wherein to extrapolate the total traffic level of the cluster, the processing device is to:

apply a statistical analysis or machine learning model on the total traffic level of the cluster during the first time interval to estimate a future total traffic level of the cluster at the end of the third time interval.

13. A non-transitory computer-readable storage medium including instructions that, when executed by a processing device, cause the processing device to:

monitor, during a first time interval, traffic associated with one or more applications executed by a cluster of compute nodes, wherein each compute node of the cluster of compute nodes comprises a virtual machine;

scale a number of replicas of the one or more applications based on the traffic associated with the one or more applications over a second time interval, the second time interval corresponding to an amount of time associated with instantiating a replica of the one or more applications;

determine, by the processing device, in view of the traffic associated with the one or more applications during the first time interval, that the traffic is predicted to exceed a capacity threshold of the cluster of compute nodes at an end of a third time interval, wherein the third time interval is longer than the second time interval; and

initiate startup of an additional compute node comprising an additional virtual machine to be added to the cluster of compute nodes for executing replicas of the one or more applications based on the traffic being predicted to exceed the capacity threshold of the cluster of compute nodes at the end of the third time interval, wherein the third time interval corresponds to an amount of time associated with starting up the additional compute node.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the one or more applications comprise one or more serverless applications.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the traffic comprises a number of concurrent requests received by each of the one or more applications executed by the cluster of compute nodes.

16. The non-transitory computer-readable storage medium of claim 13 , wherein to monitor the traffic associated with the one or more applications executed by the cluster of compute nodes, the processing device is to:

scrape one or more traffic metrics from each of the one or more applications; and

determine a total traffic level of the cluster during the first time interval in view of the one or more traffic metrics from each of the one or more applications.

17. The non-transitory computer-readable storage medium of claim 16 , wherein to determine that the traffic of the one or more applications is predicted to exceed the capacity threshold of the cluster of compute nodes at the end of the third time interval, the processing device is to:

extrapolate the total traffic level of the cluster during the first time interval over the third time interval.