Continuous learning system for models without pipelines
An ML root model format having a root model definition, is converted into an ML target model format not having the root model definition. A learning system is assigned to the root model definition that is convertible to the machine learning target model format. A new version of the root model definition is ported from the ML root model to the format of the ML target model to generate a new version of the machine learning target model after a learning iteration of the learning system. Quality limits are determined using an X control chart method based on a cross-validation of fold results generated during a validation of the new version of the machine learning root model. A quality metric value of the new version of the ML target model is evaluated against the quality threshold values of the new version of the ML root model.
1. A computer-implemented method for converting a machine learning root model format, the root model format having a related root model definition, into a machine learning target model format not having the root model definition, the method comprising:
assigning a learning system to the root model definition, wherein the machine learning root model is convertible to a format compatible with the machine learning target model;
porting a new version of the root model definition from the machine learning root model to the format of the machine learning target model to generate a new version of the machine learning target model, after a learning iteration of the learning system that generates a new version of the machine learning root model;
determining quality limits using an X control chart method based on a cross-validation of fold results generated during a validation of the new version of the machine learning root model, wherein the validation uses k folds of training data for validating the new version of the machine learning root model resulting in k quality metric values, wherein determining the quality limits comprises determining an upper control limit (UCL) of UCL=average (k quality metric values)+R*A2, and wherein R=max (k quality metric values)−min (k quality metric values), and A2=correction constant; and
evaluating a quality metric value of the new version of the machine learning target model against quality threshold values of the new version of the machine learning root model, the quality threshold values being equal to the determined quality limits.
2. The method of claim 1 , wherein the machine learning root model comprises at least one selected out of the machine learning frameworks keras, scikit-learn, xgboost, and caffe.
3. The method of claim 1 , further comprising:
deploying the machine learning target model only if a determined quality parameter value of the machine learning target model stays within the determined quality limits of the new version of the machine learning root model.
4. The method of claim 1 , wherein the determination of quality limits further comprises:
determining a lower control limit LCL of LCL=average (k quality metric values)−R*A2, wherein R=max (k quality metric values)−min (k quality metric values), and A2=correction constant.
5. The method of claim 1 , wherein a determined quality parameter is determined by building an average value of the k quality metric values.
6. The method of claim 5 , wherein the same k folds of training data are used to determine the quality metric value of the new version of the machine learning target model and the new version of the machine learning root model.
7. The method of claim 1 , further comprising:
generating an alert indicative of the fact that either the quality metric value of the new version of the machine learning root model and/or the quality metric value of the new version of the machine learning target model stay outside the quality limits.
8. A converting system for converting a machine learning root model format, the root model format having a related root model definition, into a machine learning target model format not having the root model definition, the system comprising:
a processor; and
a computer-readable storage medium communicatively coupled to the processor and storing program instructions which, when executed by the processor, cause the processor to perform a method comprising:
assigning a learning system to the root model definition, wherein the machine learning root model is convertible to a format compatible with the machine learning target model;
porting a new version of the root model definition from the machine learning root model to the format of the machine learning target model to generate a new version of the machine learning target model, after a learning iteration of the learning system that generates a new version of the machine learning root model;
determining quality limits using an X control chart method based on a cross-validation of fold results generated during a validation of the new version of the machine learning root model, wherein the validation uses k folds of training data for validating the new version of the machine learning root model resulting in k quality metric values, wherein determining the quality limits comprises determining a lower control limit (LCL) of LCL=average (k quality metric values)−R*A2, and wherein R=max (k quality metric values)−min (k quality metric values), and A2=correction constant; and
evaluating a quality metric value of the new version of the machine learning target model against the quality threshold values of the new version of the machine learning root model, the quality threshold values being equal to the determined quality limits.
9. The converting system of claim 8 , wherein the machine learning root model comprises at least one selected out of the machine learning frameworks keras, scikit-learn, xgboost, and caffe.
10. The converting system of claim 8 , wherein the method performed by the processor further comprises:
deploying the machine learning target model only if a determined quality parameter value of the machine learning target model stays within the determined quality limits of the new version of machine learning root model.
11. The converting system of claim 8 , wherein the determination of the quality limits further comprises:
determining an upper control limit UCL of UCL=average (k quality metric values)+R*A2, wherein R=max (k quality metric values)−min (k quality metric values), and A2=correction constant.
12. The system of claim 8 , wherein a determined quality parameter is determined by building an average value of the k quality metric values.
13. The system of claim 12 , wherein the same k folds of training data are used to determine the quality of the new version of the machine learning target model and the new version of the machine learning root model.
14. The system of claim 8 , wherein the method performed by the processor further comprises:
generating an alert indicative of the fact that either the quality metric value of the new version of the machine learning root model and/or the quality metric value of the new version of the machine learning target model stay outside the quality limits.
15. A computer program product for converting a machine learning root model format, the root model format having a related root model definition, into a machine learning target model format not having the root model definition, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions being executable by one or more computing systems or controllers to cause the one or more computing systems to:
assign a learning system to the root model definition, wherein the machine learning root model is convertible to a format compatible with the machine learning target model;
port a new version of the root model definition from the machine learning root model to the format of the machine learning target model to generate a new version of the machine learning target model, after a learning iteration of the learning system that generates a new version of the machine learning root model;
determine quality limits using an X control chart method based on a cross-validation of fold results generated during a validation of the new version of the machine learning root model, wherein the validation uses k folds of training data for validating the new version of the machine learning root model resulting in k quality metric values, wherein determining the quality limits comprises determining an upper control limit (UCL) of UCL=average (k quality metric values)+R*A2, and wherein R=max (k quality metric values)−min (k quality metric values), and A2=correction constant; and
evaluate a quality metric value of the new version of the machine learning target model against the quality threshold values of the new version of the machine learning root model, the quality threshold values being equal to the determined quality limits.