Utilizing masked autoencoder generative models to extract microscopy representation autoencoder embeddings
The present disclosure relates to systems, non-transitory computer-readable media, and methods for training and utilizing generative machine learning models to generate embeddings from phenomic images (or other microscopy representations). For example, the disclosed systems can train a generative machine learning model (e.g., a masked autoencoder generative model) to generate predicted (or reconstructed) phenomic images from masked version of ground truth training phenomic images. In some cases, the disclosed systems utilize a momentum-tracking optimizer while reducing a loss of the generative machine learning model to enable efficient training on large scale training image batches. Furthermore, the disclosed systems can utilize Fourier transformation losses with multi-stage weighting to improve the accuracy of the generative machine learning model on the phenomic images during training. Indeed, the disclosed systems can utilize the trained generative machine learning model to generate phenomic embeddings from input phenomic images (for various phenomic comparisons).
1. A computer-implemented method comprising:
generating a masked training phenomic image by applying a mask to a training phenomic image portraying a cell phenotype; and
training a generative machine learning model to generate phenomic perturbation image embeddings from the training phenomic image by:
generating, utilizing the generative machine learning model, a predicted phenomic image from the masked training phenomic image;
generating a Fourier transformation loss between the predicted phenomic image and the training phenomic image; and
modifying parameters of the generative machine learning model utilizing the Fourier transformation loss and a first Fourier weight.
2. The computer-implemented method of claim 1 , further comprising training the generative machine learning model utilizing a reconstruction loss between the predicted phenomic image and the training phenomic image.
3. The computer-implemented method of claim 1 , further comprising training the generative machine learning model by reducing the Fourier transformation loss utilizing a momentum-tracking optimizer.
4. The computer-implemented method of claim 1 , further comprising generating an additional masked training phenomic image by applying an additional mask to an additional training phenomic image portraying an additional cell phenotype.
5. The computer-implemented method of claim 4 , further comprising training the generative machine learning model to generate the phenomic perturbation image embeddings by generating an additional measure of loss between an additional predicted phenomic image, from the generative machine learning model, and the additional training phenomic image.
6. The computer-implemented method of claim 5 , further comprising
generating the additional measure of loss by generating an additional Fourier transformation loss between the additional predicted phenomic image and the additional training phenomic image.
7. The computer-implemented method of claim 6 , further comprising training the generative machine learning model by modifying the parameters of the generative machine learning model utilizing the additional Fourier transformation loss and a second Fourier weight.
8. The computer-implemented method of claim 7 , wherein the first Fourier weight comprises a texture Fourier weight and the second Fourier weight comprises a sharpness Fourier weight and further comprising:
utilizing the texture Fourier weight to train the generative machine learning model to improve texture recovery; and
utilizing the sharpness Fourier weight to train the generative machine learning model to improve sharpness.
9. The computer-implemented method of claim 1 , further comprising training the generative machine learning model by training a masked autoencoder generative model to generate the phenomic perturbation image embeddings.
10. The computer-implemented method of claim 9 , further comprising training the masked autoencoder generative model utilizing multiple channel modalities from the masked training phenomic image.
11. A system comprising:
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
generate a masked training phenomic image by applying a mask to a training phenomic image portraying a cell phenotype; and
train a generative machine learning model to generate perturbation image embeddings from the training phenomic image by:
generating, utilizing the generative machine learning model, a predicted phenomic image from the masked training phenomic image;
generating a Fourier transformation loss between the predicted phenomic image and the training phenomic image; and
modifying parameters of the generative machine learning model utilizing the Fourier transformation loss and a first Fourier weight.
12. The system of claim 11 , wherein the instructions cause the system to train the generative machine learning model by training a masked autoencoder generative model to generate the phenomic perturbation image embeddings utilizing multiple channel modalities from the masked training phenomic image.
13. The system of claim 11 , wherein the instructions cause the system to train the generative machine learning model by reducing the Fourier transformation loss utilizing a momentum-tracking optimizer.
14. The system of claim 1 , wherein the instructions cause the system to generate an additional masked training phenomic image by applying an additional mask to an additional training phenomic image portraying an additional cell phenotype.
15. The system of claim 14 , wherein the instructions cause the system to:
train the generative machine learning model to generate phenomic perturbation image embeddings by:
generating an additional measure of loss between an additional predicted phenomic image, from the generative machine learning model, and the additional training phenomic image by generating an additional Fourier transformation loss between the additional predicted phenomic image and the additional training phenomic image; and
training the generative machine learning model by modifying the parameters of the generative machine learning model utilizing the additional Fourier transformation loss and a second Fourier weight.
16. The system of claim 15 , wherein the first Fourier weight comprises a texture Fourier weight and the second Fourier weight comprises a sharpness Fourier weight and wherein the instructions cause the system to:
utilize the texture Fourier weight to train the generative machine learning model to improve texture recovery; and
utilize the sharpness Fourier weight to train the generative machine learning model to improve sharpness.
17. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computing device to:
generate a masked training phenomic image by applying a mask to a training phenomic image portraying a cell phenotype; and
train a generative machine learning model to generate phenomic perturbation image embeddings from the training phenomic image by:
generating, utilizing the generative machine learning model, a predicted phenomic image from the masked training microscopy phenomic image;
generating a Fourier transformation loss between the predicted phenomic image and the training microscopy phenomic image; and
modifying parameters of the generative machine learning model utilizing the Fourier transformation loss and a first Fourier weight.
18. The non-transitory computer-readable medium of claim 17 , wherein the instructions cause the computing device to train the generative machine learning model by training a masked autoencoder generative model to generate the phenomic perturbation image embeddings utilizing multiple channel modalities from the masked training phenomic image.
19. The non-transitory computer-readable medium of claim 17 , wherein the instructions cause the computing device to train the generative machine learning model by reducing the Fourier transformation loss utilizing a momentum-tracking optimizer.
20. The non-transitory computer-readable medium of claim 19 , wherein the instructions cause the computing device to:
train, in a first training iteration, the generative machine learning model to generate the phenomic perturbation image embeddings by modifying the parameters of the generative machine learning model utilizing a first Fourier weight; and
train, in a second training iteration, the generative machine learning model by modifying the parameters of the generative machine learning model utilizing a second Fourier weight.