Biological image transformation using machine-learning models
Described are systems and methods for training a machine-learning model to generate image of biological samples, and systems and methods for generating enhanced images of biological samples. The method for training a machine-learning model to generate images of biological samples may include obtaining a plurality of training images comprising a training image of a first type, and a training image of a second type. The method may also include generating, based on the training image of the first type, a plurality of wavelet coefficients using the machine-learning model; generating, based on the plurality of wavelet coefficients, a synthetic image of the second type; comparing the synthetic image of the second type with the training image of the second type; and updating the machine-learning model based on the comparison.
1. A method for generating enhanced images of biological samples, comprising:
obtaining, using a microscope, an image of a biological sample; and
generating, based on the image, an enhanced image of the biological sample using a machine-learning model, wherein the machine-learning model has been trained by:
obtaining a plurality of training images comprising
a training image of a first type, and
a training image of a second type;
generating, based on the training image of the first type, a plurality of wavelet coefficients using the machine-learning model;
generating, based on the plurality of wavelet coefficients, a synthetic image of the second type;
comparing the synthetic image of the second type with the training image of the second type; and
updating the machine-learning model based on the comparison.
2. The method of claim 1 , wherein the training image of the first type is a bright-field image of a biological sample.
3. The method of claim 2 , further comprising: generating, based on the training image of the first type, an image of a third type.
4. The method of claim 3 , wherein the image of the third type is a phase shift image.
5. The method of claim 2 , further comprising: generating, based on the training image of the first type, an image of a fourth type.
6. The method of claim 5 , wherein the image of the fourth type comprises segmentation data.
7. The method of claim 2 , wherein the training image of the first type is captured using a microscope according to a first illumination scheme.
8. The method of claim 7 , wherein the first illumination scheme comprises one or more illumination patterns.
9. The method of claim 7 , wherein the training image of the first type is part of a bright-field image array.
10. The method of claim 7 , wherein the plurality of training images is a first plurality of training images, the method further comprising:
based on the comparison, identifying a second illumination scheme;
obtaining a second plurality of training images comprising one or more images of the first type, wherein the one or more images of the first type are obtained based on the second illumination scheme;
training the machine-learning model based on the second plurality of training images.
11. The method of claim 7 , further comprising:
obtaining, using a microscope, a plurality of images of the first type; and
generating, based on the obtained plurality of images, a plurality of synthetic images of the second type using the machine-learning model.
12. The method of claim 11 , further comprising: training a classifier based on the plurality of synthetic images of the second type.
13. The method of claim 12 , wherein the microscope is a first microscope, wherein the classifier is a first classifier, further comprising:
obtaining, using a second microscope, a plurality of images of the second type;
training a second classifier based on the plurality of images of the second type;
comparing performance of the first classifier and the second classifier.
14. The method of claim 13 , wherein the second microscope is a fluorescence microscope.
15. The method of claim 1 , wherein the training image of the second type is a fluorescence image of the biological sample.
16. The method of claim 1 , wherein the machine-learning model comprises a generator and a discriminator.
17. The method of claim 16 , wherein the machine-learning model comprises a conditional GAN model.
18. The method of claim 17 , wherein the discriminator is a PatchGAN neural network.
19. The method of claim 16 , wherein the generator comprises a plurality of neural networks corresponding to a plurality of frequency groups.
20. The method of claim 19 , wherein each neural network of the plurality of neural networks is configured to generate wavelet coefficients for a respective frequency group.
21. The method of claim 19 , wherein the plurality of neural networks comprises a plurality of U-Net neural networks.
22. The method of claim 1 , wherein the image of the biological sample is illuminated using an illumination pattern of an illumination source, wherein the illumination pattern is determined by:
training a classification model configured to receive an input image and output a classification result,
training, using the trained classification model, the machine-learning model having a plurality of weights corresponding to a plurality of illumination settings, and
identifying the illumination pattern based on the plurality of weights of the trained machine-learning model.
23. The method of claim 22 , wherein the illumination source comprises an array of illumination emitters.
24. The method of claim 23 , wherein the illumination source is a LED array.
25. The method of claim 23 , wherein each illumination setting of the plurality of illumination settings corresponds to a respective illumination emitter of the illumination source; and wherein each weight corresponds to an intensity of the respective illumination emitter.
26. The method of claim 22 , wherein the illumination pattern indicates whether each illumination emitter is turned on or off and the intensity of each illumination emitter.
27. The method of claim 22 , wherein the classification model is configured to receive an input phase image or an input fluorescence image and output a classification result indicative of one class out of a plurality of pre-defined classes.
28. A method for training a machine-learning model to generate images of biological samples, comprising:
obtaining a plurality of training images comprising:
a microscopic training image of a first type, and
a microscopic training image of a second type;
generating, based on the training microscopic image of the first type, a plurality of wavelet coefficients using the machine-learning model;
generating, based on the plurality of wavelet coefficients, a synthetic image of the second type;
comparing the synthetic image of the second type with the microscopic training image of the second type; and
updating the machine-learning model based on the comparison.