Machine learning service
A machine learning service implements programmatic interfaces for a variety of operations on several entity types, such as data sources, statistics, feature processing recipes, models, and aliases. A first request to perform an operation on an instance of a particular entity type is received, and a first job corresponding to the requested operation is inserted in a job queue. Prior to the completion of the first job, a second request to perform another operation is received, where the second operation depends on a result of the operation represented by the first job. A second job, indicating a dependency on the first job, is stored in the job queue. The second job is initiated when the first job completes.
1. A computer implemented method comprising:
receiving, via a programmatic interface of a network-accessible machine learning service of a provider network, a data source creation request from a client specifying an address for a data set;
responsive to the data source creation request:
generating and storing a data source artifact corresponding to the address;
generating statistics of data retrieved from data source artifact; and
providing the statistics to the client;
receiving, via the programmatic interface, a model execution request from the client to execute a model, the model execution request specifying the data source artifact as input data and a type of output of the model; and
responsive to the model execution request:
instantiating the model on a set of resources in the provider network; and
executing the model using data retrieved from data source artifact and storing the model's output.
2. The computer implemented method of claim 1 , wherein the data source creation request specifies a format or schema of data records in the data set.
3. The computer implemented method of claim 1 , wherein:
the data source creation request specifies a sampling, splitting, or rearrangement of data records in the data set; and
the generating of the statistics is performed according to the sampling, splitting, or rearrangement.
4. The computer implemented method of claim 1 , providing the statistics to the client comprises:
providing, in a first phase, initial statistics determined from a sub-sample of the data set; and
providing, in a second phase, full-sample statistics determined from the data set in its entirety.
5. The computer implemented method of claim 1 , further comprising:
receiving, via the programmatic interface, a statistics creation request specifying the data source artifact and a statistics descriptor indicating one or more types of statistics to generate; and
responsive to the statistics creation request, generating and storing the one or more types of statistics indicated in the statistics descriptor for the data source artifact.
6. The computer implemented method of claim 1 , wherein:
the model execution request's output type specifies an evaluation execution; and
executing the model comprises determining and storing one or more accuracy measures of the model based at least in part on the model's output.
7. The computer implemented method of claim 1 , wherein:
the model execution request specifies a data transformation recipe; and
executing the model comprises applying the data transformation recipe to data from the data source artifact to generate the input data for the model.
8. The computer implemented method of claim 1 , wherein:
the model execution request specifies an expected workload level for the model; and
further comprising selecting the set of resources in the provider network for executing the model based at least in part on the expected workload level.
9. The computer implemented method of claim 1 , further comprising:
receiving, via the programmatic interface, a model publishing request to publish the model under an alias; and
responsive to the model publishing request:
mounting the model in an online execution mode; and
publishing the alias to permit the model to be executed via the alias.
10. The computer implemented method of claim 9 , further comprising:
responsive to the model execution request, generating a notification indicating that the model associated with the alias has changed since a last execution via the alias.
11. The computer implemented method of claim 1 , further comprising:
receiving, via the programmatic interface, a client request to obtain the model specifying an execution architecture for the model; and
responsive to the client request:
generating an executable representation of the model for the execution architecture and providing the executable representation.
12. A system, comprising:
one or more hardware processors with associated memory that implement a network-accessible machine learning service, configured to:
receive, via a programmatic interface of the machine learning service, a data source creation request from a client specifying an address for a data set;
responsive to the data source creation request:
generate and store a data source artifact corresponding to the address;
generate statistics of data retrieved from data source artifact; and
provide the statistics to the client;
receive, via the programmatic interface, a model execution request from the client to execute a model, the model execution request specifying the data source artifact as input data and a type of output of the model; and
responsive to the model execution request:
instantiate the model on a set of resources in the machine learning service; and
execute the model using data retrieved from data source artifact and store the model's output.
13. The system of 12 , wherein the programmatic interface comprises a representational state transfer (REST) implemented based on a hypertext transfer protocol (HTTP) protocol.
14. The system of 12 , wherein the programmatic interface is configured to be invoked via one or more software development kit (SDK) libraries.
15. The system of claim 12 , wherein the data source creation request specifies a format or schema of data records in the data set.
16. The system of claim 12 , wherein to provide the statistics to the client, the machine learning service is configured to:
provide, in a first phase, initial statistics determined from a sub-sample of the data set; and
provide, in a second phase, full-sample statistics determined from the data set in its entirety.
17. The system of claim 12 , wherein the machine learning service is configured to:
receive, via the programmatic interface, a statistics creation request specifying the data source artifact and a statistics descriptor indicating one or more types of statistics to generate; and
responsive to the statistics creation request, generate and store the one or more types of statistics indicated in the statistics descriptor for the data source artifact.
18. One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors implementing a machine learning service, cause the machine learning service to:
receive, via a programmatic interface of the machine learning service, a data source creation request from a client specifying an address for a data set;
responsive to the data source creation request:
generate and store a data source artifact corresponding to the address;
generate statistics of data retrieved from data source artifact; and
provide the statistics to the client;
receive, via the programmatic interface, a model execution request from the client to execute a model, the model execution request specifying the data source artifact as input data and a type of output of the model; and
responsive to the model execution request:
instantiate the model on a set of resources in the machine learning service; and
execute the model using data retrieved from data source artifact and store the model's output.
19. The one or more non-transitory computer-accessible storage media of claim 18 , wherein:
the model execution request specifies a data transformation recipe; and
to execute the model, the program instructions when executed on or across the one or more cause the machine learning service to apply the data transformation recipe to data from the data source artifact to generate the input data for the model.
20. The one or more non-transitory computer-accessible storage media of claim 18 , wherein:
the model execution request specifies an expected workload level for the model; and
the program instructions when executed on or across the one or more cause the machine learning service to select the set of resources in the machine learning service for executing the model based at least in part on the expected workload level.