Reuse of machine learning models
A method performed by a processing system including at least one processor includes defining a proposal for a proposed machine learning model, identifying an existing machine learning model, where the existing machine learning model shares a similarity with the proposed machine learning model, evaluating a fitness of the existing machine learning model for reuse in building the proposed machine learning model, building a new machine learning model that is consistent with the proposal for the proposed machine learning model by reusing a portion of the existing machine learning model, and monitoring a performance of the new machine learning model in a deployment environment.
1 . A method comprising:
defining, by a processing system including at least one processor, a proposal for a proposed machine learning model to be built;
identifying, by the processing system, an existing machine learning model, wherein the existing machine learning model shares a similarity with the proposed machine learning model;
evaluating, by the processing system, a fitness of the existing machine learning model for reuse in building the proposed machine learning model;
building, by the processing system, a new machine learning model that is consistent with the proposed machine learning model by reusing a portion of the existing machine learning model; and
updating, by the processing system, the new machine learning model subsequent to the new machine learning model being deployed in a deployment environment, wherein the updating is performed in response to a bias being detected in input features used to train the existing machine learning model, wherein the updating includes retraining the new machine learning model with new input features, and wherein the updating improves a quality of an output produced by the new machine learning model by mitigating the bias.
2 . The method of claim 1 , wherein the proposal for the proposed machine learning model defines at least one of: a deployment need for the proposed machine learning model, data features that the proposed machine learning model will take as input, a target of the proposed machine learning model, a performance criterion of the proposed machine learning model, a source domain of the proposed machine learning model, or a target domain of the proposed machine learning model.
3 . The method of claim 1 , wherein the proposal for the proposed machine learning model is defined with inputs received from a human user.
4 . The method of claim 1 , wherein the similarity comprises a similarity in data feature inputs.
5 . The method of claim 1 , wherein the similarity comprises a similarity in target outputs.
6 . The method of claim 1 , wherein the similarity is detected when metadata associated with the existing machine learning model matches metadata associated with the proposal for the proposed machine learning model.
7 . The method of claim 6 , wherein the metadata associated with the existing machine learning model identically matches metadata associated with the proposal for the proposed machine learning model.
8 . The method of claim 6 , wherein the metadata associated with the existing machine learning model semantically or conceptually matches metadata associated with the proposal for the proposed machine learning model.
9 . The method of claim 6 , wherein a match between the metadata associated with the existing machine learning model and the metadata associated with the proposal for the proposed machine learning model is assigned a score, and the identifying comprises determining that the score at least meets a predefined threshold score.
10 . The method of claim 1 , wherein the evaluating further comprises validating input features of the existing machine learning model to measure at least one of: a drift or the bias.
11 . The method of claim 1 , wherein the evaluating further comprises evaluating an alignment of predictions generated by the existing machine learning model with predictions that the proposed machine learning model are intended to generate.
12 . The method of claim 1 , wherein the evaluating further comprises retraining the existing machine learning model.
13 . The method of claim 1 , wherein the evaluating further comprises ranking a plurality of existing machine learning models, including the existing machine learning model which is identified, according to fitness for reuse in building the proposed machine learning model.
14 . The method of claim 1 , wherein the evaluating further comprises breaking apart the existing machine learning model and presenting examples of inputs and outputs of the existing machine learning model to a human user for manual review.
15 . The method of claim 1 , wherein the updating further comprises reporting an instance of a mismatch between a feature of the proposal for the proposed machine learning model and a feature of the new machine learning model in the deployment environment.
16 . The method of claim 1 , further comprising:
storing, by the processing system, the new machine learning model along with metadata that describes at least one of: input features of the new machine learning model, output features of the new machine learning model, a performance metric of the new machine learning model, user feedback provided during building and deployment of the new machine learning model, or portions of the existing machine learning model which were used to build the new machine learning model.
17 . The method of claim 1 , wherein the updating uses a portion of at least one of: the existing machine learning model, or another existing machine learning model.
18 . A non-transitory computer-readable medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising:
defining a proposal for a proposed machine learning model to be built;
identifying an existing machine learning model, wherein the existing machine learning model shares a similarity with the proposed machine learning model;
evaluating a fitness of the existing machine learning model for reuse in building the proposed machine learning model;
building a new machine learning model that is consistent with the proposal for the proposed machine learning model by reusing a portion of the existing machine learning model; and
updating the new machine learning model subsequent to the new machine learning model being deployed in a deployment environment, wherein the updating is performed in response to a bias being detected in input features used to train the existing machine learning model, wherein the updating includes retraining the new machine learning model with new input features, and wherein the updating improves a quality of an output produced by the new machine learning model by mitigating the bias.
19 . A device comprising:
a processing system including at least one processor; and
a non-transitory computer-readable medium storing instructions which, when executed by the processing system, cause the processing system to perform operations, the operations comprising:
defining a proposal for a proposed machine learning model to be built;
identifying an existing machine learning model, wherein the existing machine learning model shares a similarity with the proposed machine learning model;
evaluating a fitness of the existing machine learning model for reuse in building the proposed machine learning model;
building a new machine learning model that is consistent with the proposal for the proposed machine learning model by reusing a portion of the existing machine learning model; and
updating the new machine learning model subsequent to the new machine learning model being deployed in a deployment environment, wherein the updating is performed in response to a bias being detected in input features used to train the existing machine learning model, wherein the updating includes retraining the new machine learning model with new input features, and wherein the updating improves a quality of an output produced by the new machine learning model by mitigating the bias.
20 . The device of claim 19 , wherein the proposal for the proposed machine learning model defines at least one of: a deployment need for the proposed machine learning model, data features that the proposed machine learning model will take as input, a target of the proposed machine learning model, a performance criterion of the proposed machine learning model, a source domain of the proposed machine learning model, or a target domain of the proposed machine learning model.