Microservice throttling based on learned demand predictions
A dynamically-managed microservices platform. The microservices platform is configured to selectively accept admission of microservices and to selectively throttle microservices based on a continually-trained machine learning model. The system is configured to implement multiple microservice lifecycle strategies, where individual ones of the multiple microservice lifecycle strategies use a prediction model having long-term and short term demand predictions. One or another of the multiple microservice lifecycle strategies are invoked at different times under different conditions. Under a first set of conditions, a prediction model is used to select a first strategy to admit or throttle a microservice based upon long-term or short-term demand predictions. Under a second set of conditions, the prediction model is used to select a different strategy to admit or throttle a microservice. The prediction model is continually trained such that new sets of observations and new sets of predicted outputs are codified into the machine learning model.
1. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor cause a set of acts comprising:
monitoring a system having a plurality of microservices;
determining that at least one of the plurality of microservices are to be throttled;
generating short-term and long-term demand predictions for the plurality of microservices; and
analyzing at least the short-term and long-term demand predictions to identify a microservice of the plurality of microservices to throttle, wherein respective long-term demand predictions correspond to a first time range that is longer than a second time range of a corresponding short-term demand predictions.
2. The non-transitory computer readable medium of claim 1 , wherein admission of at least one microservice of the plurality of microservices was determined based on a long-term or short-term demand prediction from at least one prediction model used to select a first or a second admission strategy.
3. The non-transitory computer readable medium of claim 2 , wherein the admission of at least one microservice of the plurality of microservices was determined based on both the long-term and the short-term demand prediction from at least one prediction model used to select a first or a second admission strategy.
4. The non-transitory computer readable medium of claim 1 , wherein the set of acts further comprise throttling a non-critical microservice when either the short-term demand prediction of the non-critical microservice is high or when the long-term demand prediction of the non-critical microservice is high.
5. The non-transitory computer readable medium of claim 1 , wherein input signals to the short-term and long-term demand predictions are derived from observed behaviors comprising at least one of, CPU demands, network services demands, or storage facilities demands of the system, the system comprising multiple computing nodes.
6. The non-transitory computer readable medium of claim 1 , wherein the set of acts further comprise forming a candidate set of non-critical microservices by accessing the system to identify a set of microservices hosted on the system, and eliminating microservices that are associated with a critical label from the candidate set.
7. The non-transitory computer readable medium of claim 1 , wherein additional microservices of the plurality of microservices are throttled until the microservices are operating below a threshold.
8. A method comprising:
monitoring a system having a plurality of microservices;
determining that at least one of the plurality of microservices are to be throttled;
generating short-term and long-term demand predictions for the plurality of microservices; and
analyzing at least the short-term and long-term demand predictions to identify a microservice of the plurality of microservices to throttle, wherein respective long-term demand predictions correspond to a first time range that is longer than a second time range of a corresponding short-term demand predictions.
9. The method of claim 8 , wherein admission of at least one microservice of the plurality of microservices was determined based on a long-term or short-term demand prediction from at least one prediction model used to select a first or a second admission strategy.
10. The method of claim 9 , wherein the admission of at least one microservice of the plurality of microservices was determined based on both the long-term and the short-term demand prediction from at least one prediction model used to select a first or a second admission strategy.
11. The method of claim 8 , further comprising throttling a non-critical microservice when either the short-term demand prediction of the non-critical microservice is high or when the long-term demand prediction of the non-critical microservice is high.
12. The method of claim 8 , wherein input signals to the short-term and long-term demand predictions are derived from observed behaviors comprising at least one of, CPU demands, network services demands, or storage facilities demands of the system, the system comprising multiple computing nodes.
13. The method of claim 8 , further comprising forming a candidate set of non-critical microservices by accessing the system to identify a set of microservices hosted on the system, and eliminating microservices that are associated with a critical label from the candidate set.
14. The method of claim 8 , wherein additional microservices of the plurality of microservices are throttled until the microservices are operating below a threshold.
15. A system comprising:
a storage medium having stored thereon a sequence of instructions; and
a processor that executes the sequence of instructions to cause a set of acts comprising:
monitoring a computing cluster having a plurality of microservices;
determining that at least one of the plurality of microservices are to be throttled;
generating short-term and long-term demand predictions for the plurality of microservices; and
analyzing at least the short-term and long-term demand predictions to identify a microservice of the plurality of microservices to throttle, wherein respective long-term demand predictions correspond to a first time range that is longer than a second time range of a corresponding short-term demand predictions.
16. The system of claim 15 , wherein admission of at least one microservice of the plurality of microservices was determined based on a long-term or short-term demand prediction from at least one prediction model used to select a first or a second admission strategy.
17. The system of claim 16 , wherein the admission of at least one microservice of the plurality of microservices was determined based on both the long-term and the short-term demand prediction from at least one prediction model used to select a first or a second admission strategy.
18. The system of claim 15 , wherein the set of acts further comprise throttling a non-critical microservice when either the short-term demand prediction of the non-critical microservice is high or when the long-term demand prediction of the non-critical microservice is high.
19. The system of claim 15 , wherein input signals to the short-term and long-term demand predictions are derived from observed behaviors comprising at least one of, CPU demands, network services demands, or storage facilities demands of the computing cluster, the computing cluster comprising multiple computing nodes.
20. The system of claim 15 , wherein the set of acts further comprise forming a candidate set of non-critical microservices by accessing the computing cluster to identify a set of microservices hosted on the computing cluster, and eliminating microservices that are associated with a critical label from the candidate set.
21. The system of claim 15 , wherein additional microservices of the plurality of microservices are throttled until the microservices are operating below a threshold.
22. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor cause a set of acts comprising:
monitoring a system having a plurality of microservices;
determining that at least one of the plurality of microservices are to be throttled;
generating short-term and long-term demand predictions for the plurality of microservices; and
analyzing at least the short-term and long-term demand predictions to identify a microservice of the plurality of microservices to throttle; and
throttling a non-critical microservice when either the short-term demand prediction of the non-critical microservice is high or when the long-term demand prediction of the non-critical microservice is high.
23. The non-transitory computer readable medium of claim 22 , wherein admission of at least one microservice of the plurality of microservices was determined based on both the long-term and short-term demand prediction from at least one prediction model used to select a first or a second admission strategy.
24. The non-transitory computer readable medium of claim 22 , wherein input signals to the short-term and the long-term demand predictions are derived from observed behaviors comprising at least one of, CPU demands, network services demands, or storage facilities demands of the system, the system comprising multiple computing nodes.
25. The non-transitory computer readable medium of claim 22 , wherein additional microservices of the plurality of microservices are throttled until the microservices are operating below a threshold.
26. A method comprising:
monitoring a system having a plurality of microservices;
determining that at least one of the plurality of microservices are to be throttled;
generating short-term and long-term demand predictions for the plurality of microservices; and
analyzing at least the short-term and long-term demand predictions to identify a microservice of the plurality of microservices to throttle, wherein admission of at least one microservice of the plurality of microservices was determined based on both a long-term and a short-term demand prediction from at least one prediction model used to select a first or a second admission strategy.
27. The method of claim 26 , further comprising throttling a non-critical microservice when either the short-term demand prediction of the non-critical microservice is high or when the long-term demand prediction of the non-critical microservice is high.
28. The method of claim 26 , wherein input signals to the short-term and long-term demand predictions are derived from observed behaviors comprising at least one of, CPU demands, network services demands, or storage facilities demands of the system, the system comprising multiple computing nodes.
29. The method of claim 26 , wherein additional microservices of the plurality of microservices are throttled until the microservices are operating below a threshold.