Registry enhancements for just-in-time compilation of machine learning models

1 . A method of dynamic model compilation, comprising:

receiving, from a client, a request for a machine learning model, wherein the request indicates either:

one or more attributes comprising one or more of:

a hardware characteristic;

a target precision; or

a compiler characteristic;

one or more default behaviors to be used to compile the machine learning model;

determining that a pre-compiled model corresponding to the one or more attributes is not present in a registry;

determining a compiler for the machine learning model based on the one or more attributes or the one or more default behaviors, wherein the compiler is stored in a the registry;

compiling the machine learning model using the compiler; and

providing the compiled machine learning model to the client in response to the request.

2 . The method of claim 1 , further comprising storing the compiled machine learning model in the registry in association with metadata that is based on the one or more attributes or the one or more default behaviors.

3 . The method of claim 2 , further comprising:

receiving an additional request from an additional client specifying at least a subset of the one or more attributes; and

providing the compiled machine learning model to the additional client in response to the additional request based on the metadata.

4 . The method of claim 2 , further comprising:

storing a plurality of policies in the registry; and

based on one or more of the stored policies, removing or updating the stored, compiled machine learning model from the registry.

5 . The method of claim 1 , further comprising determining that there is no compiler corresponding to the one or more attributes in the registry, wherein the compiler is determined based on one or more default parameters.

6 . The method of claim 1 , further comprising:

determining that an updated version of the compiler has been added to the registry; and

re-compiling the machine learning model using the updated version of the compiler.

7 . The method of claim 6 , further comprising one or more of:

providing the re-compiled machine learning model to the client; or

storing the re-compiled machine learning model in the registry in association with metadata that is based on the one or more attributes or the one or more default behaviors.

8 . A system for dynamic model compilation, the system comprising:

at least one memory; and

at least one processor coupled to the at least one memory, the at least one processor and the at least one memory configured to:

receive, from a client, a request for a machine learning model, wherein the request indicates either:

one or more attributes comprising one or more of:

a hardware characteristic;

a target precision; or a compiler characteristic;

one or more default behaviors to be used to compile the machine learning model;

determine that a pre-compiled model corresponding to the one or more attributes is not present in a registry;

determine a compiler for the machine learning model based on the one or more attributes or the one or more default behaviors, wherein the compiler is stored in the registry;

compile the machine learning model using the compiler; and

provide the compiled machine learning model to the client in response to the request.

9 . The system of claim 8 , wherein the at least one processor and the at least one memory are further configured to store the compiled machine learning model in the registry in association with metadata that is based on the one or more attributes or the one or more default behaviors.

10 . The system of claim 9 , wherein the at least one processor and the at least one memory are further configured to:

receive an additional request from an additional client specifying at least a subset of the one or more attributes; and

provide the compiled machine learning model to the additional client in response to the additional request based on the metadata.

11 . The system of claim 9 , wherein the at least one processor and the at least one memory are further configured to:

store a plurality of policies in the registry; and

based on one or more of the stored policies, remove or update the stored, compiled machine learning model from the registry.

12 . The system of claim 8 , wherein the at least one processor and the at least one memory are further configured to determine that there is no compiler corresponding to the one or more attributes in the registry, wherein the compiler is determined based on one or more default parameters.

13 . The system of claim 8 , wherein the at least one processor and the at least one memory are further configured to:

determine that an updated version of the compiler has been added to the registry; and

re-compile the machine learning model using the updated version of the compiler.

14 . The system of claim 13 , wherein the at least one processor and the at least one memory are further configured to perform one or more of:

providing the re-compiled machine learning model to the client; or

storing the re-compiled machine learning model in the registry in association with metadata that is based on the one or more attributes or the one or more default behaviors.

15 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:

receive, from a client, a request for a machine learning model, wherein the request indicates either:

one or more attributes comprising one or more of:

a hardware characteristic;

a target precision; or a compiler characteristic;

one or more default behaviors to be used to compile the machine learning model;

determine that a pre-compiled model corresponding to the one or more attributes is not present in a registry;

determine a compiler for the machine learning model based on the one or more attributes or the one or more default behaviors, wherein the compiler is stored in the registry;

compile the machine learning model using the compiler; and

provide the compiled machine learning model to the client in response to the request.

16 . The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to store the compiled machine learning model in the registry in association with metadata that is based on the one or more attributes or the one or more default behaviors.

17 . The non-transitory computer-readable medium of claim 16 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

receive an additional request from an additional client specifying at least a subset of the one or more attributes; and

provide the compiled machine learning model to the additional client in response to the additional request based on the metadata.

18 . The non-transitory computer-readable medium of claim 16 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

store a plurality of policies in the registry; and

based on one or more of the stored policies, remove or update the stored, compiled machine learning model from the registry.

19 . The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to determine that there is no compiler corresponding to the one or more attributes in the registry, wherein the compiler is determined based on one or more default parameters.

20 . The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:

determine that an updated version of the compiler has been added to the registry; and

re-compile the machine learning model using the updated version of the compiler.