Machine learning based function testing
A method for determining the performance metric of a function may include interpolating the performance metric of the function relative to a known performance metric of a reference function. The performance metric of the function may be interpolated based on a first difference in a performance of the function measured by applying a first machine learning model and a performance of the function measured by applying a second machine learning model. The performance metric of the function may be further interpolated based on a second difference in a performance of the reference function measured by applying the first machine learning model and a performance of the reference function measured by applying the second machine learning model. The function may be deployed to a production system if the performance metric of the function exceeds a threshold value. Related systems and articles of manufacture, including computer program products, are also provided.
1. A system, comprising:
at least one processor; and
at least one memory including program code which when executed by the at least one processor provides operations comprising:
training a first machine learning model and a second machine learning model by at least processing, with the first machine learning model and the second machine learning model, training data that includes a first record known to be associated with a correct label and a second record known to be associated with an incorrect label;
determining, based at least on a difference between a first quantity of mislabeled records and a second quantity of mislabeled records, a performance metric for a function, the first quantity of mislabeled records determined by applying the first machine learning model to an output of the function, and the second quantity of mislabeled records determined by applying the second machine learning model to the output of the function; and
deploying, based at least on the performance metric of the function, the function to a production system.
2. The system of claim 1 , wherein the function is a filter function configured to be applied to a dataset to determine a class membership for each record of a plurality of records in the dataset.
3. The system of claim 2 , wherein the filter function is a trained machine learning model.
4. The system of claim 2 , wherein the filter function is a deterministic model.
5. The system of claim 2 , wherein the plurality of records are associated with one of a plant class or an animal class.
6. The system of claim 1 , wherein the first machine learning model and the second machine learning model are a predictive machine learning model including one or more of a support vector machine, a Bayes classifier, a neural network, a gradient boosting machine, an elastic net, or a decision tree.
7. The system of claim 1 , wherein the function is deployed to the production system when the performance metric exceeds a threshold value.
8. A method comprising:
receiving, from a client device, a request for a performance metric for a function;
training a first machine learning model and a second machine learning model using training data that includes a first record associated with a correct label and a second record associated with an incorrect label;
determining, based on a difference between a first quantity of mislabeled records and a second quantity of mislabeled records, the performance metric for the function, the first quantity of mislabeled records determined by applying the first machine learning model to an output of the function, and the second quantity of mislabeled records determined by applying the second machine learning model to the output of the function;
generating a command to cause a user interface of the client device to display the performance metric; and
sending the command to the client device.
9. The method of claim 8 , wherein the client device is one of a smartphone, a tablet computer, a wearable apparatus, a desktop computer, a laptop computer, or a workstation.
10. The method of claim 8 , wherein the function is a filter function configured to be applied to a dataset to determine a class membership for each record of a plurality of records in the dataset.
11. The method of claim 10 , wherein the filter function is a trained machine learning model.
12. The method of claim 10 , wherein the filter function is a deterministic model.
13. The method of claim 10 , wherein the plurality of records are associated with one of a plant class or an animal class.
14. The method of claim 8 , wherein the first machine learning model and the second machine learning model are a predictive machine learning model including one or more of a support vector machine, a Bayes classifier, a neural network, a gradient boosting machine, an elastic net, or a decision tree.
15. The method of claim 8 , wherein the function is deployed to a production system when the performance metric exceeds a threshold value.
16. A non-transitory computer readable medium storing instructions, which when executed by at least one data processor, result in operations comprising:
training a first machine learning model and a second machine learning model using training data that includes a first record associated with a correct label and a second record associated with an incorrect label;
determining, based on a difference between a first quantity of mislabeled records and a second quantity of mislabeled records, a performance metric for a function, the first quantity of mislabeled records determined by applying the first machine learning model to an output of the function, and the second quantity of mislabeled records determined by applying the second machine learning model to the output of the function;
generating a command to cause a user interface of a client device to display the performance metric; and
deploying, based at least on the performance metric of the function, the function to a production system.
17. The non-transitory computer readable medium storing instructions of claim 16 , wherein the client device is one of a smartphone, a tablet computer, a wearable apparatus, a desktop computer, a laptop computer, or a workstation.
18. The non-transitory computer readable medium storing instructions of claim 16 , wherein the function is a filter function configured to be applied to a dataset to determine a class membership for each record of a plurality of records in the dataset.
19. The non-transitory computer readable medium storing instructions of claim 18 , wherein the filter function is one of a trained machine learning model or a deterministic model.
20. The non-transitory computer readable medium storing instructions of claim 16 , wherein the function is deployed to a production system when the performance metric exceeds a threshold value.