Dynamically adjusting a serverless execution container pool for training and utilizing online machine-learning models
The disclosure describes one or more implementations of a serverless computing management system that utilizes an online learning model to dynamically adjust the number of serverless execution containers in a serverless pool based on incoming data patterns. For example, for each time instance in a given time period, the serverless computing management system utilizes the online learning model to balance computing latency and computing cost to determine how to intelligently resize the serverless pool, such that the online machine-learning models in the serverless pool can update in a manner that improves accuracy and computing efficiency while also minimizing unnecessary delays. Further, the serverless computing management system provides a framework that facilitates state-based training of online machine-learning models in a stateless and serverless cloud-based environment.
1. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computer system to:
monitor arrival patterns of machine learning model data over a time period comprising a series of time instances;
determine a computing latency and a computing cost of processing the machine learning model data utilizing a current number of serverless execution containers from a pool of serverless execution containers for the time period;
utilize an online learning model to determine a second number of serverless execution containers by:
generating, utilizing parameters of the online learning model, a first adjusted computing latency and a first adjusted computing cost for a first time instance of the series of time instances by modeling a change from the current number of serverless execution containers to the second number of serverless execution containers in the pool of serverless execution containers at the first time instance;
generating, utilizing the parameters of the online learning model, a second adjusted computing latency and a second adjusted computing cost for a second time instance of the series of time instances by modeling a change from the current number of serverless execution containers to the second number of serverless execution containers in the pool of serverless execution containers at the second time instance; and
selecting the second number of serverless execution containers by comparing the computing latency, the computing cost, the first adjusted computing latency, the first adjusted computing cost, the second adjusted computing latency, and the second adjusted computing cost;
identify additional machine learning model data after the time period; and
provide the additional machine learning model data to the second number of serverless execution containers in the pool of serverless execution containers.
2. The non-transitory computer-readable medium of claim 1 , wherein the instructions that, when executed by the at least one processor, cause the computer system to determine the second number of serverless execution containers by determining that changing a number of serverless execution containers from the current number of serverless execution containers to the second number of serverless execution containers is within a regret threshold for a current time instance.
3. The non-transitory computer-readable medium of claim 2 , wherein the instructions that, when executed by the at least one processor, cause the computer system to determine that the second number of serverless execution containers is within the regret threshold by:
estimating a computing latency change and a computing cost change resulting from utilizing the second number of serverless execution containers to process the arrival patterns of the machine learning model data across the time period; and
comparing the computing latency change and the computing cost change.
4. The non-transitory computer-readable medium of claim 3 , further comprising instructions that, when executed by the at least one processor, cause the computer system to determine the second number of serverless execution containers further by weighting the computing cost change by a parameter indicating a prioritization of the computing cost to the computing latency, wherein selecting the second number of serverless execution containers is further based on comparing the computing latency change to the weighted computing cost change.
5. The non-transitory computer-readable medium of claim 2 , wherein the instructions that, when executed by the at least one processor, cause the computer system to determine that the second number of serverless execution containers is within the regret threshold by:
determining a computing latency change resulting from utilizing the second number of serverless execution containers to process the arrival patterns of the machine learning model data across the time period; and
comparing the computing latency change to a maximum latency increase threshold.
6. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computer system to select the second number of serverless execution containers by:
estimating, utilizing the parameters of the online learning model, a computing latency change between the computing latency and one of the first adjusted computing latency or the second adjusted computing latency;
estimating, utilizing the parameters of the online learning model, a computing cost change between the computing cost and one of the first adjusted computing cost or the second adjusted computing cost;
comparing the computing latency change to the computing cost change and a maximum latency increase threshold; and
in response to determining that the computing latency change exceeds the computing cost change, adding one or more serverless execution containers to the current number of serverless execution containers within the pool of serverless execution containers; or
in response to determining that the computing latency change does not exceed the maximum latency increase threshold, removing one or more serverless execution containers from the current number of serverless execution containers within the pool of serverless execution containers.
7. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computer system to determine the computing cost of processing the machine learning model data utilizing the current number of serverless execution containers by applying a serverless execution container provisioning cost to the current number of serverless execution containers.
8. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
provide the machine learning model data and machine learning model parameters for one or more machine learning models to the current number of serverless execution containers;
determine modified machine learning model parameters for the one or more machine learning models via the current number of serverless execution containers; and
store the modified machine learning model parameters in a shared storage medium.
9. The non-transitory computer-readable medium of claim 8 , wherein the one or more machine learning models comprise a neural network, a decision tree, bandit-based model, a logical regression model, or a support vector machine.
10. The non-transitory computer-readable medium of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
identify the additional machine learning model data and the modified machine learning model parameters in the shared storage medium; and
provide the additional machine learning model data and the modified machine learning model parameters to the second number of serverless execution containers in the pool of serverless execution containers for determining further modified parameters of the one or more machine learning models.
11. A system comprising:
a memory device comprising machine learning model data and an online learning model for dynamically sizing a pool of serverless execution containers; and
at least one computing device configured to cause the system to:
monitor arrival patterns of the machine learning model data over a time period comprising a series of time instances;
provide the machine learning model data to a current number of serverless execution containers for determining modified parameters of one or more machine learning models;
determine a computing latency and a computing cost of processing the machine learning model data utilizing the current number of serverless execution containers from the pool of serverless execution containers for the time period;
determine a second number of serverless execution containers utilizing the online learning model by:
generating, utilizing parameters of the online learning model, a first adjusted computing latency and a first adjusted computing cost for a first time instance of the series of time instances by modeling a change from the current number of serverless execution containers to the second number of serverless execution containers in the pool of serverless execution containers at the first time instance;
estimating a computing latency change by comparing the computing latency with the first adjusted computing latency and a computing cost change by comparing the computing cost with the first adjusted computing cost; and
selecting the second number of serverless execution containers by comparing the computing latency change and the computing cost change; and
provide additional machine learning model data to the second number of serverless execution containers in the pool of serverless execution containers.
12. The system of claim 11 , wherein the at least one computing device is further configured to cause the system to:
provide the machine learning model data and machine learning model parameters to the one or more machine learning models for processing via the current number of serverless execution containers;
determine modified machine learning model parameters for the one or more machine learning models via the current number of serverless execution containers; and
store the modified machine learning model parameters in a shared storage medium.
13. The system of claim 12 , wherein the at least one computing device is further configured to cause the system to:
identify the modified machine learning model parameters in the shared storage medium; and
provide the modified machine learning model parameters to the second number of serverless execution containers in the pool of serverless execution containers in connection with providing the additional machine learning model data to the second number of serverless execution containers.
14. The system of claim 11 , wherein the online learning model comprises a neural network, a decision tree, bandit-based model, a logical regression model, or a support vector machine.
15. The system of claim 11 , wherein the at least one computing device is further configured to cause the system to determine the second number of serverless execution containers by:
estimating the computing latency change and the computing cost change resulting from utilizing the second number of serverless execution containers to process the arrival patterns of the machine learning model data; and
selecting the second number of serverless execution containers by comparing the computing latency change and the computing cost change, wherein the second number of serverless execution containers is greater than the current number of serverless execution containers.
16. The system of claim 11 , wherein the at least one computing device is further configured to cause the system to determine the second number of serverless execution containers by:
estimating the computing latency change resulting from utilizing the second number of serverless execution containers to process the arrival patterns of the machine learning model data; and
selecting the second number of serverless execution containers by comparing the computing latency change to a maximum latency increase limit, wherein the second number of serverless execution containers is less than the current number of serverless execution containers.
17. In a digital serverless environment for continuously updating online machine-learning models, a computer-implemented method for dynamically adjusting a plurality of serverless execution containers, comprising:
monitoring arrival patterns of machine learning model data over a time period comprising a series of time instance;
providing the machine learning model data and machine learning model parameters for one or more machine learning models to a current number of serverless execution containers in a pool of serverless execution containers;
determining modified machine learning model parameters for the one or more machine learning models via the current number of serverless execution containers;
determining a computing latency and a computing cost of processing the machine learning model data utilizing the current number of serverless execution containers from the pool of serverless execution containers for the time period;
determining a second number of serverless execution containers in the pool of serverless execution containers that maximizes computing latency reduction by:
generating, utilizing parameters of the online machine-learning models, a first adjusted computing latency and a first adjusted computing cost for a first time instance of the series of time instances by modeling a change from the current number of serverless execution containers to the second number of serverless execution containers in the pool of serverless execution containers at the first time instance;
estimating a computing latency change by comparing the computing latency with the first adjusted computing latency and a computing cost change by comparing the computing cost with the first adjusted computing cost; and
selecting the second number of serverless execution containers by comparing the computing latency change and the computing cost change; and
providing additional machine learning model data and the modified machine learning model parameters to the second number of serverless execution containers.
18. The computer-implemented method of claim 17 , further comprising;
storing the modified machine learning model parameters at a shared storage medium; and
identifying, before providing the modified machine learning model parameters to the second number of serverless execution containers, the modified machine learning model parameters at the shared storage medium.
19. The computer-implemented method of claim 17 , further comprising provisioning new serverless execution containers within the second number of serverless execution containers with software and packages corresponding to the one or more machine learning models before providing the additional machine learning model data to the second number of serverless execution containers.
20. The computer-implemented method of claim 17 ,
wherein the second number of serverless execution containers is greater than the current number of serverless execution containers.