System and method for coordinated resource scaling in microservice-based and serverless applications
A computer-implemented method for trace-driven call-graph-aware proactive coordinated autoscaling of component microservices in an application includes generating performance-resource elasticity models of endpoints of the component microservices of the application. Workload levels of the endpoint of the component microservices is predicted based on user traffic observed at a front end service. A trace-level performance of the application is predicted for different microservice replica scaling based on the performance-resource elasticity models at end points, the ends points on the trace call graph and the predicted workload levels. A microservice replica scaling is recommended for each of the component microservices to meet predefined trace-level user service level objectives.
1 . A computer-implemented method for trace-driven call-graph-aware proactive coordinated autoscaling of component microservices in an application, the method comprising:
generating performance-resource elasticity models of endpoints of the component microservices of the application;
predicting workload levels of the endpoint of the component microservices based on user traffic observed at a front end service;
predicting a trace-level performance of the application for different microservice replica scaling based on the performance-resource elasticity models of the component microservice endpoints, the end points on a trace call graph and the predicted workload levels; and
recommending a microservice replica scaling for each of the component microservices to meet predefined trace-level user service level objectives.
2 . The computer-implemented method of claim 1 , wherein the recommended microservice replica scaling is based on a microservice endpoint self-latency aggregation.
3 . The computer-implemented method of claim 2 , wherein the microservice endpoint self-latency is defined as an isolated latency contribution of a given endpoint of a given component microservice to an overall trace latency for each component microservice used in the application.
4 . The computer-implemented method of claim 1 , wherein the trace-level service level objectives include at least one of latency or throughput targets.
5 . The computer-implemented method of claim 1 , wherein the performance-resource elasticity models predict self-latency performance at each of the endpoints of a given component microservice as a function of a vector of workload levels to all endpoints that belong to the given microservice.
6 . The computer-implemented method of claim 1 , wherein the predicted trace-level performance is based on an aggregated self-latency of the component endpoints identified in a trace call graph.
7 . The computer-implemented method of claim 1 , further comprising using a machine learning model for generating the performance-resource elasticity models.
8 . The computer-implemented method of claim 1 , wherein the predicted workload levels are based on a predicted load, a currently observed load, or a combination thereof.
9 . The computer-implemented method of claim 1 , further comprising predicting workload levels at each endpoints of the component microservice used in the application.
10 . The computer-implemented method of claim 1 , further comprising using a machine learning model for generating the predicted workload levels.
11 . The computer-implemented method of claim 1 , further comprising leveraging mixed-integer programming scaling optimization for predicting the trace-level performance of the application for different microservice replica scaling.
12 . The computer-implemented method of claim 1 , further comprising learning a pattern of cascading calls to predict workload levels across multiple traces.
13 . A system comprising:
a processor;
a memory coupled to the processor; and
a computer readable storage embodying a computer program code, the computer program code comprising instructions for trace-driven call-graph-aware proactive coordinated autoscaling of component microservices in an application, the instructions executable by the processor and configured to:
generate performance-resource elasticity models of endpoints of the component microservices of the application;
predict workload levels of the endpoints of the component microservices based on user traffic observed at a front end service;
predict a trace-level performance of the application for different microservice replica scaling based on the performance-resource elasticity models and the predicted workload levels; and
recommend a microservice replica scaling for each of the component microservices to meet predefined trace-level user service level objectives.
14 . The system of claim 13 , wherein:
the recommended microservice replica scaling is based on a microservice endpoint self-latency aggregation; and
the microservice endpoint self-latency is defined as an isolated latency contribution of a given endpoint of a given component microservice to an overall trace latency for each component microservice used in the application.
15 . The system of claim 13 , wherein the trace-level service level objectives include at least one of latency or throughput targets.
16 . The system of claim 13 , wherein:
the performance-resource elasticity models predict self-latency performance at each of the endpoints of any given component microservice as a function of a vector of workload levels to all endpoints that belong to the given microservice; and
the predicted trace-level performance is based on an aggregated self-latency of the component microservices identified in a trace graph.
17 . The system of claim 13 , wherein the instructions are further configured to:
use a machine learning model for generating the performance-resource elasticity models; and
use a machine learning model for generating the predicted workload levels.
18 . The system of claim 13 , wherein the instructions are further configured to predict workload levels at each endpoints of the component microservice used in the application, wherein the predicted workload levels are based on a predicted load, a currently observed load, or a combination thereof.
19 . A computer program product for trace-driven call-graph-aware proactive coordinated autoscaling of component microservices in an application, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:
generate performance-resource elasticity models of endpoints of the component microservices of the application;
predict workload levels of the endpoint of the component microservices based on user traffic observed at a front end service;
predict a trace-level performance of the application for different microservice replica scaling based on the performance-resource elasticity models and the predicted workload levels; and
recommend a microservice replica scaling for each of the component microservices to meet predefined trace-level user service level objectives.
20 . The computer program product of claim 19 , wherein:
the recommended microservice replica scaling is based on a microservice endpoint self-latency aggregation; and
the microservice endpoint self-latency is defined as an isolated latency contribution of a given endpoint of a given component microservice to an overall trace latency for each component microservice used in the application.