Automated drift detection in multidimensional data
View Patent ↗Disclosed embodiments provide techniques for automated drift detection in multidimensional data. Disclosed embodiments utilize deep learning to estimate high-density regions of multidimensional, multivariate, and/or multimodal data and combine the results with hypothesis testing. A hypothesis of drift or no drift is tested using a mathematical test, and if drift is detected, mitigation actions, including retraining are performed, to enable the continuation of reliable, high-quality results from the neural network.
1 . A computer-implemented method for monitoring performance of a neural network, comprising:
obtaining input data, wherein the input data comprises multidimensional data;
estimating at least one high-density region of the input data, wherein a high-density region is a data region of the input data that has a data volume value that exceeds a minimum data volume value threshold;
performing a statistical drift detection test on the at least one high-density region; and
in response to detecting a drift, executing at least one mitigation action, wherein the at least one mitigation action includes:
temporarily pausing the neural network, the pausing preventing the neural network from evaluating new input data during a retraining phase of the neural network;
retraining, while the neural network is temporarily paused, the neural network using the new input data; and
automatically unpausing the retrained neural network after the retraining has been completed.
2 . The method of claim 1 , wherein the drift detection test comprises a Kolmogorov-Smirnov test.
3 . The method of claim 1 , wherein the drift detection test comprises a Chi-squared test.
4 . The method of claim 1 , wherein the at least one mitigation action includes issuing a data drift alert.
5 . The method of claim 1 , wherein detecting the drift comprises detecting a data drift.
6 . The method of claim 1 , wherein detecting the drift comprises detecting a concept drift.
7 . The method of claim 1 , wherein estimating the at least one high-density region is performed with a second neural network.
8 . The method of claim 7 , wherein the second neural network comprises an autoencoder.
9 . The method of claim 7 , wherein the second neural network comprises a generative adversarial network.
10 . The method of claim 9 , wherein the second neural network comprises a CycleGAN neural network.
11 . The method of claim 1 , wherein the input data comprises multivariate data.
12 . The method of claim 11 , wherein the input data comprises multimodal data.
13 . An electronic computation device comprising:
a processor; and
a memory coupled to the processor, the memory containing instructions, that when executed by the processor, cause the electronic computation device to:
obtain input data, wherein the input data comprises multidimensional data;
estimate at least one high-density region of the input data, wherein a high-density region is a data region of the input data that has a data volume value that exceeds a minimum data volume value threshold;
perform a statistical drift detection test on the at least one high-density region; and
in response to detecting a drift, execute at least one mitigation action, wherein the at least one mitigation action includes:
temporarily pausing a neural network, the pausing preventing the neural network from evaluating new input data during a retraining phase of the neural network;
retraining, while the neural network is temporarily paused, the neural network using the new input data; and
automatically unpausing the retrained neural network after the retraining has been completed.
14 . The electronic computation device of claim 13 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to perform the drift detection test using a Kolmogorov-Smirnov test.
15 . The electronic computation device of claim 13 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to perform the drift detection test using a Chi-squared test.
16 . The electronic computation device of claim 13 , wherein the memory further comprises instructions, that when executed by the processor, cause the electronic computation device to estimate the at least one high-density region using a second neural network.
17 . A computer program product for an electronic computation device comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the electronic computation device to:
obtain input data, wherein the input data comprises multidimensional data;
estimate at least one high-density region of the input data, wherein a high-density region is a data region of the input data that has a data volume value that exceeds a minimum data volume value threshold;
perform a statistical drift detection test on the at least one high-density region; and
in response to detecting a drift, execute at least one mitigation action, wherein the at least one mitigation action includes:
temporarily pausing the neural network, the pausing preventing the neural network from evaluating new input data during a retraining phase of the neural network;
retraining, while the neural network is temporarily paused, the neural network using the new input data; and
automatically unpausing the retrained neural network after the retraining has been completed.
18 . The computer program product of claim 17 , wherein the computer readable storage medium further comprises program instructions, that when executed by the processor, cause the electronic computation device to perform the drift detection test using a Kolmogorov-Smirnov test.
19 . The computer program product of claim 17 , wherein the computer readable storage medium further comprises program instructions, that when executed by the processor, cause the electronic computation device to estimate the at least one high-density region using a second neural network.