Automated processing of multiple prediction generation including model tuning

1. A method, comprising:

receiving, from a client device, a dataset comprising a plurality of rows, a row including values for a set of columns, and the set of columns including at least one or more keys with one or more key-values in the dataset;

analyzing the dataset to determine a dimension along one or more keys;

determining two or more groupings of the one or more key-values in the dataset for the dimension in the dataset, wherein the two or more groupings include a first grouping with a first set of key-values for the one or more keys, and a second grouping with a second set of key-values for the one or more keys that are different from the first grouping, wherein each of the first grouping and the second grouping is a unique set of key-values;

training a set of machine-learned models, wherein each machine-learned model to train corresponds to a respective unique grouping for the dimension, comprising:

obtaining a respective subset of the dataset for training a first machine-learned model for the first grouping and another respective subset of the dataset for training a second machine-learned model for the second grouping;

storing the subset of the dataset for the first machine-learned model in a cache for a compute resource and storing the another subset of the dataset for the second machine-learned model in a cache for another compute resource; and

training the first machine-learned model and the second machine-learned model through a machine-learned process with the respective subsets of the dataset in parallel using the compute resource and the another compute resource;

receiving, using an application programming interface (API), a request for prediction on a query dataset, the request including one or more prediction parameters for the prediction;

selecting a machine-learned model from the set of machine-learned models based on the key-values of the query dataset of the request, wherein the grouping for the selected machine-learned model matches the key-values of the dimension for the query dataset; and

applying the selected model to the query dataset to generate a prediction and providing the prediction to the client device as a response.

2. The method of claim 1 , further comprising:

exposing the set of machine-learned models to the client device as a composite model via an application programming interface (API) or a web interface, and

wherein the request for prediction is received through the API or the web interface.

3. The method of claim 1 , further comprising:

responsive to a triggering condition, updating parameters of a machine-learned model in the set of machine-learned models; and

updating a composite model to include or refer to the updated machine-learned model.

4. The method of claim 3 , further comprising:

obtaining historical information pertaining to training of the machine-learned model, wherein the historical information includes a set of parameters previously trained for the machine-learned model, and

wherein updating the parameters of the machine-learned model further comprises:

determining a search space for the parameters based on the historical information, and

restricting search of the updated parameters of the machine-learned model to the search space determined from the historical information.

5. The method of claim 3 , wherein the triggering condition is at least one of receiving an updated dataset for training the machine-learned model, lapse of a predetermined interval of time from when the machine-learned model was trained, or determining that a difference between the respective subset of the dataset used to train the machine-learned model and an updated dataset for updating the machine-learned model is above a threshold.

6. The method of claim 1 , wherein each grouping of key-values in the set of groupings is a combination of two or more key-values.

7. The method of claim 1 , wherein the one or more prediction parameters is at least one of a type of prediction for the query dataset, a scope of the prediction, or a temporal bounding condition of the prediction.

8. A non-transitory computer-readable storage medium comprising stored instructions executable by a processor, the instructions when executed causing the processor to:

receive, from a client device, a dataset comprising a plurality of rows, a row including values for a set of columns, and the set of columns including at least one or more keys with one or more key-values in the dataset;

analyze the dataset to determine a dimension along one or more keys;

determine two or more groupings of the one or more key-values in the dataset for the dimension in the dataset, wherein the two or more groupings include a first grouping with a first set of key-values for the one or more keys, and a second grouping with a second set of key-values for the one or more keys that are different from the first grouping, wherein each of the first grouping and the second grouping is a unique set of key-values;

training a set of machine-learned models, wherein each machine-learned model to train corresponds to a respective unique grouping for the dimension, comprising:

obtain a respective subset of the dataset for training a first machine-learned model for the first grouping and another respective subset of the dataset for training a second machine-learned model for the second grouping;

store the subset of the dataset for the first machine-learned model in a cache for a compute resource and storing the another subset of the dataset for the second machine-learned model in a cache for another compute resource; and

train the first machine-learned model and the second machine-learned model through a machine-learned process with the respective subsets of the dataset in parallel using the compute resource and the another compute resource;

receive, using an application programming interface (API), a request for prediction on a query dataset, the request including one or more prediction parameters for the prediction;

select a machine-learned model from the set of machine-learned models based on the key-values of the query dataset of the request, wherein the grouping for the selected machine-learned model matches the key-values of the dimension for the query dataset; and

apply the selected model to the query dataset to generate a prediction and providing the prediction to the client device as a response.

9. The non-transitory computer-readable storage medium of claim 8 , wherein the stored instructions further comprise instructions that when executed cause the processor to:

expose the set of machine-learned models to the client device as a composite model via an application programming interface (API) or a web interface; and

receive the request for prediction through the API or the web interface.

10. The non-transitory computer-readable storage medium of claim 8 , wherein the stored instructions further comprise instructions that when executed cause the processor to:

detect a triggering condition;

update parameters of a machine-learned model in the set of machine-learned models when there is the triggering condition is detected; and

update a composite model to include or refer to the updated machine-learned model.

11. The non-transitory computer-readable storage medium of claim 10 , the stored instructions further comprise instructions that when executed cause the processor to:

obtain historical information pertaining to training of the machine-learned model, wherein the historical information includes a set of parameters previously trained for the machine-learned model, and

the instruction to update the parameters of the machine-learned model further comprises instructions to:

determine a search space for the parameters based on the historical information, and

restrict search of the updated parameters of the machine-learned model to the search space determined from the historical information.

12. The non-transitory computer-readable storage medium of claim 10 , wherein the instructions to detect the triggering condition further comprises at least one of instructions to receive an updated dataset for training the machine-learned model, determine a lapse of a predetermined interval of time from when the machine-learned model was trained, or determine that a difference between the respective subset of the dataset used to train the machine-learned model and an updated dataset for updating the machine-learned model is above a threshold.

13. The non-transitory computer-readable storage medium of claim 8 , wherein each grouping of key-values in the set of groupings is a combination of two or more key-values.

14. The non-transitory computer-readable storage medium of claim 8 , wherein the one or more prediction parameters is at least one of a type of prediction for the query dataset, a scope of the prediction, or a temporal bounding condition of the prediction.

15. A computer system, comprising:

a computer processor; and

a non-transitory computer-readable storage medium comprising instructions that when executed by the computer processor cause the computer system to perform actions comprising:

receiving, from a client device, a dataset comprising a plurality of rows, a row including values for a set of columns, and the set of columns including at least one or more keys with one or more key-values in the dataset;

analyzing the dataset to determine a dimension along one or more keys;

determining two or more groupings of the one or more key-values in the dataset for the dimension in the dataset, wherein the two or more groupings include a first grouping with a first set of key-values for the one or more keys, and a second grouping with a second set of key-values for the one or more keys that are different from the first grouping, wherein each of the first grouping and the second grouping is a unique set of key-values;

training a set of machine-learned models, wherein each machine-learned model to train corresponds to a respective unique grouping for the dimension, comprising:

obtaining a respective subset of the dataset for training a first machine-learned model for the first grouping and another respective subset of the dataset for training a second machine-learned model for the second grouping;

storing the subset of the dataset for the first machine-learned model in a cache for a compute resource and storing the another subset of the dataset for the second machine-learned model in a cache for another compute resource; and

training the first machine-learned model and the second machine-learned model through a machine-learned process with the respective subsets of the dataset in parallel using the compute resource and the another compute resource;

receiving, using an application programming interface (API), a request for prediction on a query dataset, the request including one or more prediction parameters for the prediction;

selecting a machine-learned model from the set of machine-learned models based on the key-values of the query dataset of the request, wherein the grouping for the selected machine-learned model matches the key-values of the dimension for the query dataset; and

applying the selected model to the query dataset to generate a prediction and providing the prediction to the client device as a response.

16. The computer system of claim 15 , the instructions further comprising:

exposing the set of machine-learned models to the client device as a composite model via an application programming interface (API) or a web interface, and

wherein the request for prediction is received through the API or the web interface.

17. The computer system of claim 15 , the instructions further comprising:

responsive to a triggering condition, updating parameters of a machine-learned model in the set of machine-learned models; and

updating a composite model to include or refer to the updated machine-learned model.

18. The computer system of claim 17 , the instructions further comprising:

obtaining historical information pertaining to training of the machine-learned model, wherein the historical information includes a set of parameters previously trained for the machine-learned model, and

wherein updating the parameters of the machine-learned model further comprises:

determining a search space for the parameters based on the historical information, and

restricting search of the updated parameters of the machine-learned model to the search space determined from the historical information.

19. The computer system of claim 17 , wherein the triggering condition is at least one of receiving an updated dataset for training the machine-learned model, lapse of a predetermined interval of time from when the machine-learned model was trained, or determining that a difference between the respective subset of the dataset used to train the machine-learned model and an updated dataset for updating the machine-learned model is above a threshold.

20. The computer system of claim 15 , wherein the one or more prediction parameters is at least one of a type of prediction for the query dataset, a scope of the prediction, or a temporal bounding condition of the prediction.