Method, apparatus, and computer program product for machine learning model lifecycle management

1. A computing system for machine learning model lifecycle management, the computing system comprising:

application programming interface (API) circuitry configured to

receive a machine learning model selection,

receive a machine learning model experiment creation input, wherein the machine learning model experiment creation input comprises a machine learning model experiment name, a machine learning model experiment description, a machine learning model execution engine selection, at least one machine learning model scoring file, and at least one machine learning model scoring data file,

receive a machine learning model experiment run type, wherein the machine learning model experiment run type is one of batch scoring, real-time scoring, or training, and

receive a machine learning model input data path; and

machine learning model lifecycle management circuitry configured to

determine a machine learning model execution engine based on the machine learning model execution engine selection and the machine learning model experiment run type,

retrieve input data based on the machine learning model input data path,

execute a machine learning model experiment based on the machine learning model execution engine, the machine learning model scoring file, the at least one machine learning model scoring data file, and the input data, and

generate one or more machine learning model scores based on the machine learning model experiment, wherein the one or more machine learning model scores are associated with the machine learning model experiment and the input data.

2. The computing system of claim 1 , wherein the machine learning model lifecycle management circuitry is configured to:

retrieve the input data stored in a memory location identified by the machine learning model input data path;

load the at least one machine learning model scoring data file into a machine learning model execution environment based on the machine learning model execution engine selection; and

execute the machine learning model scoring file to create a machine learning model data frame comprising the one or more machine learning model scores.

3. The computing system of claim 1 , wherein the input data comprises a feature vector, wherein the feature vector comprises a most recent observation for each of a plurality of features, and wherein each of the plurality of features is identified by a feature identifier and associated with an observation timestamp.

4. The computing system of claim 1 , wherein the one or more machine learning model scores comprise one or more score statistics, and wherein each score statistic of the one or more machine learning model scores comprises a feature identifier, a score timestamp, a score, and an experiment identifier.

5. The computing system of claim 1 , further comprising publishing circuitry configured to:

receive a machine learning model experiment publishing selection; and

enable access to the machine learning model experiment and the one or more machine learning model scores based on the machine learning model experiment publishing selection.

6. The computing system of claim 5 , wherein the publishing circuitry is further configured to:

publish the machine learning model experiment and the one or more machine learning model scores in a feature catalog.

7. The computing system of claim 6 , wherein the feature catalog comprises a plurality of feature definitions.

8. The computing system of claim 1 , further comprising user interface (UI) circuitry configured to generate Flux UI display screen data.

9. The computing system of claim 1 , wherein the at least one machine learning model scoring data file comprises a trained machine learning model.

10. The computing system of claim 9 , further comprising machine learning model training circuitry configured to train the trained machine learning model using a machine learning model training dataset, wherein the machine learning model training dataset comprises a plurality of observations of a target variable and a feature vector comprising a plurality of associated observations of non-target variables observed at a time relative to the target variable.

11. The computing system of claim 1 , wherein the machine learning model experiment run type is batch scoring, wherein the determined machine learning model execution engine is a batch execution engine, and wherein the machine learning model lifecycle management circuitry is further configured to:

generate a batch execution control signal indicative of an electronic instruction to execute the machine learning model experiment based on the batch execution engine;

transmit the batch execution control signal to batch execution circuitry configured to execute the machine learning model experiment based on the batch execution engine;

receive batch execution data from the batch execution circuitry, wherein the batch execution data is indicative of one or more results of the machine learning model experiment executed by the batch execution circuitry based on the batch execution engine; and

generate the one or more machine learning model scores based on the batch execution data.

12. The computing system of claim 1 , wherein the machine learning model experiment run type is real-time scoring, wherein the determined machine learning model execution engine is a real-time execution engine, and wherein the machine learning model lifecycle management circuitry is further configured to:

generate a real-time execution control signal indicative of an electronic instruction to execute the machine learning model experiment based on the real-time execution engine;

transmit the real-time execution control signal to real-time execution circuitry configured to execute the machine learning model experiment based on the real-time execution engine;

receive real-time execution data from the real-time execution circuitry, wherein the real-time execution data is indicative of one or more results of the machine learning model experiment executed by the real-time execution circuitry based on the real-time execution engine; and

generate the one or more machine learning model scores based on the real-time execution data.

13. A computing method for machine learning model lifecycle management, the computing method comprising:

receiving, by application programming interface (API) circuitry, a machine learning model selection;

receiving, by the API circuitry, a machine learning model experiment creation input, wherein the machine learning model experiment creation input comprises a machine learning model experiment name, a machine learning model experiment description, a machine learning model execution engine selection, at least one machine learning model scoring file, and at least one machine learning model scoring data file;

receiving, by the API circuitry, a machine learning model experiment run type, wherein the machine learning model experiment run type is one of batch scoring, real-time scoring, or training;

receiving, by the API circuitry, a machine learning model input data path;

determining, by machine learning model lifecycle management circuitry, a machine learning model execution engine based on the machine learning model execution engine selection and the machine learning model experiment run type;

retrieving, by the machine learning model lifecycle management circuitry, input data based on the machine learning model input data path;

executing, by the machine learning model lifecycle management circuitry, a machine learning model experiment based on the machine learning model execution engine, the machine learning model scoring file, the at least one machine learning model scoring data file, and the input data; and

generating, by the machine learning model lifecycle management circuitry, one or more machine learning model scores based on the machine learning model experiment, wherein the one or more machine learning model scores are associated with the machine learning model experiment and the input data.

14. The computing method of claim 13 , wherein the executing the machine learning model experiment comprises:

retrieving, by the machine learning model lifecycle management circuitry, the input data stored in a memory location identified by the machine learning model input data path;

loading, by the machine learning model lifecycle management circuitry, the at least one machine learning model scoring data file into a machine learning model execution environment based on the machine learning model execution engine selection; and

executing, by the machine learning model lifecycle management circuitry, the machine learning model scoring file to create a machine learning model data frame comprising the one or more machine learning model scores.

15. The computing method of claim 13 , wherein the input data comprises a feature vector, wherein the feature vector comprises a most recent observation for each of a plurality of features, and wherein each of the plurality of features is identified by a feature identifier and associated with an observation timestamp.

16. The computing method of claim 13 , wherein the one or more machine learning model scores comprise one or more score statistics, and wherein each score statistic of the one or more machine learning model scores comprises a feature identifier, a score timestamp, a score, and an experiment identifier.

17. The computing method of claim 13 , further comprising:

receiving, by publishing circuitry, a machine learning model experiment publishing selection;

enabling, by the publishing circuitry, access to the machine learning model experiment and the one or more machine learning model scores based on the machine learning model experiment publishing selection; and

publishing, by the publishing circuitry, the machine learning model experiment and the one or more machine learning model scores in a feature catalog comprising a plurality of feature definitions.

18. The computing method of claim 13 , further comprising:

generating, by user interface (UI) circuitry, Flux UI display screen data.

19. The computing method of claim 13 , wherein the at least one machine learning model scoring data file comprises a trained machine learning model, and wherein the computing method further comprises:

training, by machine learning model training circuitry, the trained machine learning model using a machine learning model training dataset, wherein the machine learning model training dataset comprises a plurality of observations of a target variable and a feature vector comprising a plurality of associated observations of non-target variables observed at a time relative to the target variable.

20. A computer program product for machine learning model lifecycle management, the computer program product comprising at least one non-transitory computer-readable storage medium storing computer-executable program code instructions that, when executed by a computing system, cause the computing system to:

receive, by application programming interface (API) circuitry, a machine learning model selection;

receive, by the API circuitry, a machine learning model experiment creation input, wherein the machine learning model experiment creation input comprises a machine learning model experiment name, a machine learning model experiment description, a machine learning model execution engine selection, at least one machine learning model scoring file, and at least one machine learning model scoring data file;

receive, by the API circuitry, a machine learning model experiment run type, wherein the machine learning model experiment run type is one of batch scoring, real-time scoring, or training;

receive, by the API circuitry, a machine learning model input data path;

determine, by machine learning model lifecycle management circuitry, a machine learning model execution engine based on the machine learning model execution engine selection and the machine learning model experiment run type;

retrieve, by the machine learning model lifecycle management circuitry, input data based on the machine learning model input data path;

execute, by the machine learning model lifecycle management circuitry, a machine learning model experiment based on the machine learning model execution engine, the machine learning model scoring file, the at least one machine learning model scoring data file, and the input data; and

generate, by the machine learning model lifecycle management circuitry, one or more machine learning model scores based on the machine learning model experiment, wherein the one or more machine learning model scores are associated with the machine learning model experiment and the input data.