Model training using build artifacts
The subject technology detects a code commit at a code repository. The subject technology sends a request for a build job to a build server. The subject technology determines that the build job is completed. The subject technology sends a training request and user token to a proxy authenticator. The subject technology determines determining that the user token is validated. The subject technology sends a training request and the user token to a training job manager. Further, the subject technology determines determining that the training job is completed.
1. A method comprising:
receiving a request to perform a machine learning model training job;
initiating, based on a particular container image corresponding to machine learning model training, a container for performing the machine learning model training job;
retrieving a build artifact corresponding to the machine learning model training job;
performing, in the container, the machine learning model training job using the build artifact corresponding to the machine learning model training job; and
providing an indication that the machine learning model training job has been performed.
2. The method of claim 1 , wherein performing, in the container, the machine learning model training job using the build artifact corresponding to the machine learning model training job comprises:
performing, in the container, the machine learning model training job using the build artifact to generate a training artifact.
3. The method of claim 2 , further comprising storing the training artifact, wherein the indication that the machine learning model training job has been performed comprises a network identifier corresponding to the stored training artifact.
4. The method of claim 2 , wherein the training artifact comprises at least one of a serialized machine learning model or evaluation data.
5. The method of claim 1 , wherein the indication that the machine learning model training job has been performed comprises information related to an error that occurred while performing the machine learning model training job.
6. The method of claim 1 , wherein the request comprises an identifier of the build artifact, and wherein retrieving the build artifact corresponding to the machine learning model training job comprises:
retrieving the build artifact based at least in part on the identifier of the build artifact.
7. The method of claim 1 , wherein the request comprises a user token, and wherein the method further comprises:
performing an access check using the user token to determine that access to the build artifact is authorized.
8. The method of claim 1 , wherein retrieving the build artifact corresponding to the machine learning model training job comprises:
retrieving a virtual environment for performing the machine learning model training job.
9. The method of claim 8 , wherein performing, in the container, the machine learning model training job using the build artifact corresponding to the machine learning model training job comprises:
performing, in the container, the machine learning model training job in the virtual environment using the build artifact.
10. A device comprising:
a memory; and
at least one processor configured to:
receive a request to perform a machine learning model training job;
initiate, based on a particular container image corresponding to machine learning model training, a container for performing the machine learning model training job;
retrieve a build artifact corresponding to the machine learning model training job;
perform, in the container, the machine learning model training job using the build artifact corresponding to the machine learning model training job to generate a training artifact; and
store the training artifact.
11. The device of claim 10 , wherein the at least one processor is further configured to:
provide an indication that the machine learning model training job has been performed, wherein the indication comprises a network identifier corresponding to the training artifact.
12. The device of claim 10 , wherein the training artifact comprises at least one of a serialized machine learning model or evaluation data, and wherein the build artifact comprises at least one of a compressed file, an archived file, or a package.
13. The device of claim 10 , wherein the request comprises an identifier of the build artifact, and wherein the at least one processor is further configured to retrieve the build artifact based at least in part on the identifier of the build artifact.
14. The device of claim 10 , wherein the request comprises a user token, and wherein the at least one processor is further configured to:
perform an access check using the user token to determine that access to the build artifact is authorized.
15. The device of claim 10 , wherein the at least one processor is further configured to:
retrieve a virtual environment for performing the machine learning model training job; and
perform, in the container, the machine learning model training job in the virtual environment using the build artifact.
16. A non-transitory machine-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a request to perform a machine learning model training job, wherein the request comprises an identifier of a build artifact;
initiating, based on a particular container image corresponding to machine learning model training, a container for performing the machine learning model training job;
retrieving the build artifact based at least in part on the identifier of the build artifact;
performing, in a container, the machine learning model training job using the retrieved build artifact to generate a training artifact; and
performing at least one of: providing an indication that the machine learning model training job has been performed or storing the training artifact.
17. The non-transitory machine-readable medium of claim 16 , wherein the indication that the machine learning model training job has been performed comprises a network identifier corresponding to the stored training artifact.
18. The non-transitory machine-readable medium of claim 16 , wherein the training artifact comprises at least one of a serialized machine learning model or evaluation data.
19. The non-transitory machine-readable medium of claim 16 , wherein retrieving the build artifact based at least in part on the identifier of the build artifact comprises:
retrieving a virtual environment for performing the machine learning model training job.
20. The non-transitory machine-readable medium of claim 19 , wherein performing, in the container, the machine learning model training job using the retrieved build artifact to generate the training artifact comprises;
performing, in the container, the machine learning model training job in the virtual environment using the build artifact.