Drift-tolerant machine learning models

1 . A non-transitory computer-readable storage medium comprising instructions which, when executed by one or more hardware processors, causes performance of operations comprising:

determining a training data set comprising a first plurality of data points;

generating a trained machine learning model by training a machine learning algorithm using the training data set;

determining an effect of data drift on the trained machine learning model at least by:

applying at least one deterministic function to one or more numerical values in the first plurality of data points to generate a divergent data set comprising a second plurality of data points that diverges from the first plurality of data points;

obtaining a set of results by applying the trained machine learning model to the divergent data set; and

based on the set of results, determining the effect of the data drift on the trained machine learning model;

determining whether or not the trained machine learning model failed to adapt to the divergent data set based on the effect of the data drift;

based on determining whether or not the trained machine learning model failed to adapt to the divergent data set, performing one of:

generating and deploying an updated machine learning model by executing computing resources to retrain the machine learning algorithm using the divergent data set if the trained machine learning model failed to adapt to the divergent data set; and

refraining from using computing resources to generate the updated machine learning model if the trained machine learning model adapted to the divergent data set.

2 . The medium of claim 1 , wherein: the first plurality of data points correspond respectively to data points in an n-dimensional vector space, wherein applying the at least one deterministic function comprises:

identifying a region of an n-dimensional vector space that does not include any of the first plurality of data points; and

selecting the second plurality of data points from the region of the n-dimensional vector space.

3 . The medium of claim 1 , wherein generating the divergent data set is based further on (a) applying at least one non-deterministic function to the one or more numerical values in the first plurality of data points, or (b) applying the at least one non-deterministic function to an output of the at least one deterministic function.

4 . The medium of claim 1 , wherein values for a particular attribute in the first plurality of data points spans a different range of numerical values than values for the particular attribute in the second plurality of data points.

5 . The medium of claim 1 , wherein generating the divergent data set comprises:

determining one or more relationships among attributes of the data points in the first plurality of data points;

determining one or more new relationships among the attributes by modifying the one or more relationships; and

generating the divergent data set based on the one or more new relationships among the attributes.

6 . The medium of claim 1 , wherein generating the divergent data set comprises generating the second plurality of data points such that relationships between attributes in the second plurality of data points differ from relationships between attributes in the first plurality of data points.

7 . The medium of claim 1 , wherein determining the effect of the data drift comprises:

generating an alert identifying the data drift; and

the effect of data drift comprises an identification of the data drift by the trained machine learning model.

8 . The medium of claim 1 , wherein the set of results comprise predictions by the trained machine learning model corresponding to the divergent data set.

9 . The medium of claim 8 , wherein determining the effect of the data drift comprises determining an accuracy of the predictions by the trained machine learning model corresponding to the divergent data set.

10 . The medium of claim 9 , determining the effect of the data drift further comprises determining whether the trained machine learning model adapts to the divergent data set based on the accuracy of the predictions corresponding to the divergent data set.

11 . The medium of claim 9 , determining the effect of the data drift further comprises:

determining whether the trained machine learning model adapts to the divergent data set based on a difference between: the accuracy of the predictions corresponding to the divergent data set and an accuracy of predictions corresponding to the first plurality of data points.

12 . The medium of claim 1 , determining the effect of the data drift comprises:

determining a resource usage for application of the trained machine learning model on the divergent data set.

13 . The medium of claim 1 , wherein the operations further comprise:

prior to making any predictions based on the divergent data set comprising the second plurality of data points, evaluating the divergent data set for consistency with the first plurality of data points;

identifying an inconsistency between the first plurality of data points and the second plurality of data points; and

triggering a notification based on the inconsistency between the first plurality of data points and the second plurality of data points.

14 . The medium of claim 1 , wherein determining the effect of the data drift on the trained machine learning model comprises determining whether predictions by the trained machine learning model meet one or more performance criteria.

15 . A method comprising:

determining a training data set comprising a first plurality of data points;

generating a trained machine learning model by training a machine learning algorithm using the training data set;

determining an effect of data drift on the trained machine learning model at least by:

applying at least one deterministic function to one or more numerical values in the first plurality of data points to generate a divergent data set comprising a second plurality of data points that diverges from the first plurality of data points;

obtaining a set of results by applying the trained machine learning model to the divergent data set; and

based on the set of results, determining the effect of the data drift on the trained machine learning model,

determining whether or not the trained machine learning model failed to adapt to the divergent data set based on the effect of the data drift;

based on determining whether or not the trained machine learning model failed to adapt to the divergent data set, performing one of:

generating and deploying an updated machine learning model by executing computing resources to retrain the machine learning algorithm using the divergent data set if the trained machine learning model failed to adapt to the divergent data set; and

refraining from using computing resources to generate the updated machine learning model if the trained machine learning model adapted to the divergent data set,

wherein the method is performed by at least one device comprising a hardware computer processor.

16 . The method of claim 15 , wherein: the first plurality of data points correspond respectively to data points in an n-dimensional vector space, wherein applying the at least one deterministic function comprises:

identifying a region of an n-dimensional vector space that does not include any of the first plurality of data points; and

selecting the second plurality of data points from the region of the n-dimensional vector space.

17 . The method of claim 15 , wherein generating the divergent data set is based further on (a) applying at least one non-deterministic function to the one or more numerical values in the first plurality of data points, or (b) applying the at least one non-deterministic function to an output of the at least one deterministic function.

18 . The method of claim 15 , wherein values for a particular attribute in the first plurality of data points spans a different range of numerical values than values for the particular attribute in the second plurality of data points.

19 . The method of claim 15 , wherein generating the divergent data set comprises:

determining one or more relationships among attributes of the data points in the first plurality of data points;

determining one or more new relationships among the attributes by modifying the one or more relationships; and

generating the divergent data set based on the one or more new relationships among the attributes.

20 . The method of claim 15 , wherein generating the divergent data set comprises generating the second plurality of data points such that relationships between attributes in the second plurality of data points differ from relationships between attributes in the first plurality of data points.

21 . The method of claim 15 , wherein determining the effect of the data drift comprises:

generating an alert identifying the data drift; and

the effect of data drift comprises an identification of the data drift by the trained machine learning model.

22 . The method of claim 15 , wherein the set of results comprise predictions by the trained machine learning model corresponding to the divergent data set.

23 . The method of claim 22 , determining the effect of the data drift further comprises:

determining whether the trained machine learning model adapts to the divergent data set based on a difference between an accuracy of the predictions corresponding to the divergent data set and an accuracy of predictions corresponding to the first plurality of data points.

24 . The method of claim 15 , determining the effect of the data drift comprises determining a resource usage for application of the trained machine learning model on the divergent data set.

25 . The method of claim 15 , further comprising:

prior to making any predictions based on the divergent data set comprising the second plurality of data points, evaluating the divergent data set for consistency with the first plurality of data points;

identifying an inconsistency between the first plurality of data points and the second plurality of data points; and

triggering a notification based on the inconsistency between the first plurality of data points and the second plurality of data points.

26 . A system comprising one or more hardware processors and a non-transitory computer-readable medium storing program instructions that, when executed by the one or more hardware processors, causes performance of operations comprising:

determining a training data set comprising a first plurality of data points;

generating a trained machine learning model by training a machine learning algorithm using the training data set;

determining an effect of data drift on the trained machine learning model at least by:

applying at least one deterministic function to one or more numerical values in the first plurality of data points to generate a divergent data set comprising a second plurality of data points that diverges from the first plurality of data points;

obtaining a set of results by applying the trained machine learning model to the divergent data set; and

based on the set of results, determining the effect of the data drift on the trained machine learning model;

determining whether or not the trained machine learning model failed to adapt to the divergent data set based on the effect of the data drift;

based on determining whether or not the trained machine learning model failed to adapt to the divergent data set, performing one of:

generating and deploying an updated machine learning model by executing computing resources to retrain the machine learning algorithm using the divergent data set if the trained machine learning model failed to adapt to the divergent data set; and

refraining from using computing resources to generate the updated machine learning model if the trained machine learning model adapted to the divergent data set.