Image data synthesis using adversarial autoencoders for continual training of models
Systems and methods are provided for retraining machine learning (ML) models. Examples may automatically identify skewed, anomalous, and/or drift occurrence data in real-world input data. By automatically identifying such data, examples can reduce subjectivity in ML model retraining as well as reduce time spent determining a need to retrain a ML model. Accordingly, a determination can be made objectively by a computing system or device according to computer-implemented instructions. Additionally, examples may automatically isolate and transfer data relevant to the retraining of a ML model to a training environment for retraining the ML model using real-world input data. Examples also synthesize large samples of data for use in retraining a ML model. The synthesized data may be generated based on the isolated and transferred data and can be used in place of actual real-world input data to reduce a corresponding delay.
1. A system, comprising:
a processor; and
a memory configured to store instructions that, when executed by the processor, cause the processor to:
generate, for a stream of images to be processed by a model, a multi-dimensional representation of the stream of images that relates features of at least one image of the stream of images to features of at least one other image of the stream of images;
identify, based on the multi-dimensional representation, unknown images;
generate clusters of known images and at least one cluster of the unknown images based on the multi-dimensional representation of the stream of images, wherein the at least one cluster of unknown images corresponds to the unknown images;
create synthetic images based on the identified unknown images, wherein creating the synthetic images comprises:
identifying cluster densities for the cluster of unknown images;
weighting the cluster densities in proportion to a density count of the cluster densities; and
generating the synthetic images based on the weighted cluster densities of unknown images; and
retrain the model with training data including the synthetic images.
2. The system of claim 1 , wherein the multi-dimensional representation of the stream of images comprises a latent space of the stream of images generated by an encoder of the processor, the encoder processing the stream of images.
3. The system of claim 1 ,
the clusters of known images include threshold features of training images used to train the model, and
the at least one cluster of unknown images do not include the threshold features of the training images.
4. The system of claim 1 , wherein the instructions further cause the processor to apply a Gaussian Mixture model to the at least one cluster of unknown images to recreate the at least one cluster of unknown images for further processing with respect to the cluster densities.
5. The system of claim 1 , wherein the instructions further cause the processor to provide a set of unknown images to a user.
6. A method, comprising:
training a machine learning (ML) model and an adversarial autoencoder (AAE) based on a set of training images;
receiving a stream of images for processing by the ML model and the AAE, the stream of images comprising images corresponding to images of the set of training images and unknown images for which the ML model and the AAE have not been trained;
detecting the unknown images based on generating a latent space based on the stream of images, wherein detecting the unknown images comprises generating clusters of known images and at least one cluster of unknown images based on the latent space, wherein the at least one cluster of unknown images corresponds to the unknown images;
identifying a region of the latent space for use in retraining the ML model, the region of the latent space comprising at least a set of the unknown images;
generating new latent space information based on the identified region of the latent space;
generating synthesized images based on the new latent space generating new latent space information based on the identified region of the latent space comprises:
identifying cluster densities for the at least one cluster of unknown images; and
weighting the cluster densities in proportion to a density count of the cluster densities, wherein the synthesized images are generated based on the weighted cluster densities of unknown images; and
retraining the ML model based on the synthesized images.
7. The method of claim 6 , wherein the latent space is generated by an encoder of the AAE, the encoder processing the stream of images.
8. The method of claim 6 , wherein:
the clusters of known images include threshold features of training images used to train the ML model, and the at least one cluster of unknown images do not include the threshold features of the training images.
9. The method of claim 6 , wherein generating new latent space information based on the identified region of the latent space further comprises applying a Gaussian Mixture model to the at least one cluster of unknown images to recreate the at least one cluster of unknown images for further processing with respect to the cluster densities.
10. The method of claim 8 , further comprising reconstructing a set of unknown images from the stream of images using a decoder of the AAE.
11. The method of claim 10 , further comprising providing the reconstructed set of unknown images to a user.
12. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor, cause the processor to perform a method comprising:
training an AAE with a baseline based on a set of training data;
receiving a stream of data for processing by a trained ML model, the stream of data comprising data corresponding to the set of training data and unknown data for which the ML model and the AAE have not been trained;
identifying the unknown data in the stream of data based on processing the data in the stream of data, wherein identifying the unknown data comprises generating clusters of data corresponding to the set of training data and at least one cluster of unknown data, wherein the at least one cluster of unknown data corresponds to the unknown data;
generating synthesized data based on the unknown data in the stream of data, wherein generating the synthesized data comprises:
identifying cluster densities for the at least one cluster of unknown data; and
weighting the cluster densities in proportion to a density count of the cluster densities, wherein the synthesized data is further generated based on the weighted cluster densities of unknown data; and
retraining the ML model and the AAE based on the synthesized data.
13. The non-transitory computer-readable storage medium of claim 12 , including further instructions which cause the processor to perform a method further comprising:
detecting the unknown data based on generating a latent space based on the stream of data;
identifying a region of the latent space for use in retraining the ML model, the region of the latent space comprising at least a set of the unknown data; and
generating new latent space information based on the identified region of the latent space, wherein the synthesized data is generated based on the new latent space information.
14. The non-transitory computer-readable storage medium of claim 13 , wherein the latent space is generated by an encoder of the AAE processing the stream of data.
15. The non-transitory computer-readable storage medium of claim 13 , wherein generating the clusters of data corresponding to the set of training data and the at least one cluster of unknown data is based on the latent space, and wherein the clusters of known data include threshold features of training data used to train the AAE and the at least one cluster of unknown data do not include the threshold features of the training data.
16. The non-transitory computer-readable storage medium of claim 15 , wherein generating new latent space information based on the identified region of the latent space is based on the identifying of the cluster densities of unknown data, and the weighting of the cluster densities.
17. The non-transitory computer-readable storage medium of claim 16 , wherein generating new latent space information based on the identified region of the latent space further comprises applying a Gaussian Mixture model to the at least one cluster of unknown data to recreate the at least one cluster of unknown data for further processing with respect to the cluster densities.
18. The non-transitory computer-readable storage medium of claim 12 , including further instructions which cause the processor to perform a method further comprising:
reconstructing a set of unknown data from the stream of data using a decoder of the AAE; and
providing the reconstructed set of unknown data to a user.
19. The system of claim 1 , wherein the instructions further cause the processor to:
reconstruct a set of unknown images from the stream of images using a decoder; and
providing the reconstructed set of unknown images to a user.
20. The system of claim 19 , further comprising:
an adversarial autoencoder (AAE) that includes the decoder.