Active learning drift analysis and training
One or more systems, devices, computer program products and/or computer-implemented methods of use provided herein relate to training a learning model based on determined drift. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise a selection component that can select an ensemble of deep learning regressors, and an identification component that can identify drift among the ensemble. An analysis component can analyze uncertainty samplings from the ensemble to determine a time instant when drift occurred. A training component can train one or more deep learning models, such as of the deep learning regressors, based upon the identified drift.
1 . A system, comprising:
a memory that stores computer executable components; and
a processor that executes the computer executable components stored in the memory, wherein the computer executable components comprise:
a selection component that selects an ensemble of deep learning regressors respectively comprising one or more deep learning models, wherein the one or more deep learning models are trained and deployed in a computing environment accessible to users;
an identification component that identifies that the ensemble is experiencing drift, wherein identifying that the ensemble is experiencing the drift comprises determining that a majority of the deep learning regressors of the ensemble have the drift;
an analysis component that analyzes uncertainty samplings from the ensemble to determine a time period when the drift occurred, wherein the determining the time period comprises:
identifying, as a beginning of the time period, an earliest time instance when any one deep learning regressor of the majority started experiencing the drift, and
identifying, as an ending time of the time period, a latest time instance when any other deep learning regressor of the majority started experiencing the drift; and
a training component that retrains at least one deep learning model of at least one of the deep learning regressors of the majority based on the drift and the time period when the drift occurred.
2 . The system of claim 1 , wherein the identification component identifies that the ensemble is experiencing the drift based on a comparison between the uncertainty samplings, from the ensemble, having a highest uncertainty among a set of comparisons of the uncertainty samplings.
3 . The system of claim 1 , wherein the computer executable components further comprise:
a sampling component that samples one or more levels of uncertainty of the deep learning regressors of the ensemble.
4 . The system of claim 1 , wherein the training component retrains less than all deep learning models of all the deep learning regressors of the ensemble based on a selectively identified confidence interval threshold for the drift.
5 . The system of claim 1 , wherein the deep learning regressors comprise a deep neural network and a transformer that employ sequence to sequence learning.
6 . The system of claim 1 , wherein the wherein identifying the drift is further based on at least one of respective lower uncertainty bounds or respective upper uncertainty bounds for the one or more deep learning models.
7 . The system of claim 1 , wherein the identification component identifies the drift based further on respective upper uncertainty bounds for the one or more deep learning models.
8 . The system of claim 1 , wherein the identification component identifies the drift based further on respective lower uncertainty bounds for the one or more deep learning models.
9 . A computer-implemented method, comprising:
selecting, by a system operatively coupled to a processor, an ensemble of deep learning regressors respectively comprising one or more deep learning models, wherein the one or more deep learning models are trained and deployed in a computing environment accessible to users;
identifying, by the system, that the ensemble is experiencing drift, wherein identifying that the ensemble is experiencing the drift comprises determining that a majority of the deep learning regressors of the ensemble have the drift;
analyzing, by the system, uncertainty samplings from the ensemble to determine a time period when the drift occurred, wherein the determining the time period comprises:
identifying, as a beginning of the time period, an earliest time instance when any one deep learning regressor of the majority started experiencing the drift, and
identifying, as an ending time of the time period, a latest time instance when any other deep learning regressor of the majority started experiencing the drift; and
retraining, by the system, at least one deep learning model of at least one of the deep learning regressors of the majority based on the drift and the time period when the drift occurred.
10 . The computer-implemented method of claim 9 , further comprising:
identifying, by the system, that the ensemble is experiencing the drift based on a comparison between the uncertainty samplings, from the ensemble, having a highest uncertainty among a set of comparisons of the uncertainty samplings.
11 . The computer-implemented method of claim 9 , further comprising:
sampling, by the system, one or more levels of uncertainty of the deep learning regressors of the ensemble.
12 . The computer-implemented method of claim 9 , wherein the retraining comprises:
retraining, by the system, less than all deep learning models of all the deep learning regressors of the ensemble based on a selectively identified confidence interval threshold for the drift.
13 . The computer-implemented method of claim 9 , wherein the deep learning regressors comprise a deep neural network and a transformer that employ sequence to sequence learning.
14 . The computer-implemented method of claim 9 , wherein identifying the drift is further based on respective upper uncertainty bounds for the one or more deep learning models.
15 . The computer-implemented method of claim 9 , wherein identifying the drift is further based on respective lower uncertainty bounds for the one or more deep learning models.
16 . A computer program product facilitating a process to train learning models based on determined drift, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
select, by the processor, an ensemble of deep learning regressors respectively comprising one or more deep learning models, wherein the one or more deep learning models are trained and deployed in a computing environment accessible to users;
identify, by the processor, that the ensemble is experiencing drift, wherein identifying that the ensemble is experiencing the drift comprises determining that a majority of the deep learning regressors of the ensemble have the drift;
analyze, by the processor, uncertainty samplings from the ensemble to determine a time period when the drift occurred, wherein the determining the time period comprises:
identifying, as a beginning of the time period, an earliest time instance when any one deep learning regressor of the majority started experiencing the drift, and
identifying, as an ending time of the time period, a latest time instance when any other deep learning regressor of the majority started experiencing the drift; and
retrain, by the processor, at least one deep learning model of at least one of the deep learning regressors of the majority based on the drift and the time period when the drift occurred.
17 . The computer program product of claim 16 , wherein the program instructions are further executable by the processor to cause the processor to:
identify, by the processor, that the ensemble is experiencing the drift based on a comparison between the uncertainty samplings, from the ensemble, having a highest uncertainty among a set of comparisons of the uncertainty samplings.
18 . The computer program product of claim 16 , wherein the program instructions are further executable by the processor to cause the processor to:
sample, by the processor, one or more levels of uncertainty of the deep learning regressors of the ensemble.
19 . The computer program product of claim 16 , wherein the retraining comprises:
retraining less than all deep learning models of all the deep learning regressors of the ensemble based on a selectively identified confidence interval threshold for the drift.
20 . The computer program product of claim 16 , wherein the deep learning regressors comprise a deep neural network and a transformer that employ sequence to sequence learning.