Dynamically scalable machine learning model generation and retraining through containerization
In an example embodiment, a model generation component may additionally assign various cloud resources to a machine learned model so that the training or retraining of the model can be performed using these resource. The containers may be weighted to handle model generation work of different weight. Having one single configuration for a container responsible for generating all models leads to overuse of hardware resources because machine learning algorithms are very resource intensive, and thus dynamically selecting the weight improves hardware utilization.
1 . A system comprising:
at least one hardware processor; and
a computer-readable medium storing instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform operations comprising:
obtaining a dynamic weighted container assignment machine learned model trained via training using a first machine learning algorithm, the training comprising obtaining a first set of training data and passing the first set of training data through the machine learning algorithm to learn a coefficient for each of a plurality of features of the training data, the dynamic weighted container assignment machine learned model being trained to output a container configuration for a combination of an entity and an inference machine learned model, the container configuration including a category indicating a count of each of a plurality of computing resources to be assigned to the combination of the entity and the inference model;
receiving, at an application server in a cloud environment, a request to generate a first inference model for a first entity of a plurality of entities corresponding to the cloud environment;
in response to the receiving, inputting a set of features corresponding to the entity and to the first inference model to the dynamic weighted container assignment machine learned model to obtain a container configuration for the first inference model, the set of features comprising an indication of whether a type of a second machine learning algorithm is a neural network or a non-neural network, the dynamic weighted container assignment machine learning model designed to output a different container configuration if the type of the second machine learning algorithm is a neural network than if the type of the second machine learning algorithm is a non-neural network;
generating a container based on the obtained container configuration; and
causing a first version of the first inference model to be generated and trained using the container, the second machine learning algorithm, and a second set of training data.
2 . The system of claim 1 , wherein the first machine learning algorithm is a clustering algorithm.
3 . The system of claim 2 , wherein the clustering algorithm is a k-nearest neighbor algorithm.
4 . The system of claim 1 , wherein the first entity is a group of users.
5 . The system of claim 1 , wherein the operations further comprise:
receiving, at the application server, training data parameters for the first inference model and wherein the causing the first version of the first inference model to be generated and trained further includes filtering the second set of training data based on the training data parameters.
6 . The system of claim 1 , further comprising repeating the inputting and generating for a subsequent version of the first inference model, causing a different container configuration to be used for retraining of the first inference model than was used in a prior training of the first inference model.
7 . The system of claim 1 , wherein the inputting a set of features is only performed once a threshold amount of historic data of metadata about model generation runs is gathered.
8 . The system of claim 1 , wherein the set of features corresponding to the entity and to the first inference model includes information about a volume of the second set of training data.
9 . The system of claim 1 , wherein the set of features corresponding to the entity and to the first inference model includes information about a number of unique features in the second set of training data.
10 . A method comprising:
obtaining a dynamic weighted container assignment machine learned model trained via training using a first machine learning algorithm, the training comprising obtaining a first set of training data and passing the first set of training data through the machine learning algorithm to learn a coefficient for each of a plurality of features of the training data, the dynamic weighted container assignment machine learned model being trained to output a container configuration for a combination of an entity and an inference machine learned model, the container configuration including a category indicating a count of each of a plurality of computing resources to be assigned to the combination of the entity and the inference model;
receiving, at an application server in a cloud environment, a request to generate a first inference model for a first entity of a plurality of entities corresponding to the cloud environment;
in response to the receiving, inputting a set of features corresponding to the entity and to the first inference model to the dynamic weighted container assignment machine learned model to obtain a container configuration for the first inference model, the set of features comprising an indication of whether a type of a second machine learning algorithm is a neural network or a non-neural network, the dynamic weighted container assignment machine learning model designed to output a different container configuration if the type of the second machine learning algorithm is a neural network than if the type of the second machine learning algorithm is a non-neural network;
generating a container based on the obtained container configuration; and
causing a first version of the first inference model to be generated and trained using the container, the second machine learning algorithm, and a second set of training data.
11 . The method of claim 10 , wherein the first machine learning algorithm is a clustering algorithm.
12 . The method of claim 11 , wherein the clustering algorithm is a k-nearest neighbor algorithm.
13 . The method of claim 10 , wherein the first entity is a group of users.
14 . The method of claim 10 , further comprising:
receiving, at the application server, training data parameters for the first inference model and wherein the causing the first version of the first inference model to be generated and trained further includes filtering the second set of training data based on the training data parameters.
15 . A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
obtaining a dynamic weighted container assignment machine learned model trained via training using a first machine learning algorithm, the training comprising obtaining a first set of training data and passing the first set of training data through the machine learning algorithm to learn a coefficient for each of a plurality of features of the training data, the dynamic weighted container assignment machine learned model being trained to output a container configuration for a combination of an entity and an inference machine learned model, the container configuration including a category indicating a count of each of a plurality of computing resources to be assigned to the combination of the entity and the inference model;
receiving, at an application server in a cloud environment, a request to generate a first inference model for a first entity of a plurality of entities corresponding to the cloud environment;
in response to the receiving, inputting a set of features corresponding to the entity and to the first inference model to the dynamic weighted container assignment machine learned model to obtain a container configuration for the first inference model, the set of features comprising an indication of whether a type of a second machine learning algorithm is a neural network or a non-neural network, the dynamic weighted container assignment machine learning model designed to output a different container configuration if the type of the second machine learning algorithm is a neural network than if the type of the second machine learning algorithm is a non-neural network;
generating a container based on the obtained container configuration; and
causing a first version of the first inference model to be generated and trained using the container, the second machine learning algorithm, and a second set of training data.
16 . The non-transitory machine-readable medium of claim 15 , wherein the first machine learning algorithm is a clustering algorithm.
17 . The non-transitory machine-readable medium of claim 16 , wherein the clustering algorithm is a k-nearest neighbor algorithm.
18 . The non-transitory machine-readable medium of claim 15 , wherein the first entity is a group of users.
19 . The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise:
receiving, at the application server, training data parameters for the first inference model and wherein the causing the first version of the first inference model to be generated and trained further includes filtering the second set of training data based on the training data parameters.
20 . The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise:
repeating the inputting and generating for a subsequent version of the first inference model, causing a different container configuration to be used for retraining of the first inference model than was used in a prior training of the first inference model.