Deep learning model for noise reduction in low SNR imaging conditions
Embodiments disclosed herein are generally related to a system for noise reduction in low signal to noise ratio imaging conditions. A computing system obtains a set of images of a specimen. The set of images includes at least two images of the specimen. The computing system inputs the set of images of the specimen into a trained denoising model. The trained denoising model is configured to output a single denoised image of the specimen. The computing system receives, as output from the trained denoising model, a single denoised image of the specimen.
1. A system, comprising:
an imaging apparatus configured to perform darkfield imaging of a specimen; and
a computing system in communication with the imaging apparatus, the computing system comprising one or more processors and a memory, the memory having programming coded thereon, which, when executed by the one or more processors, causes the computing system to perform operations comprising:
obtaining, by the computing system, a plurality of noisy images of the specimen captured by the imaging apparatus using darkfield imaging, wherein the plurality of noisy images includes at least two images of the specimen captured using darkfield imaging;
denoising, by the computing system, the plurality of noisy images by inputting the plurality of noisy images of the specimen into a convolutional neural network trained to output a single denoised image of the specimen; and
generating, as output from the convolutional neural network, the single denoised image of the specimen.
2. The system of claim 1 , wherein the operations further comprise:
generating, by the computing system, a synthetic data set for training the convolutional neural network, the synthetic data set comprising a plurality of synthetic images and, for each synthetic image, a plurality of noisy images derived from the respective synthetic image; and
training, by the computing system, the convolutional neural network to denoise a target plurality of noisy images based on the synthetic data set.
3. The system of claim 2 , wherein the operations further comprise:
generating, by the computing system, an empirical data set for finetuning the convolutional neural network following the training; and
finetuning, by the computing system, the convolutional neural network based on the empirical data set.
4. The system of claim 3 , wherein generating, by the computing system, the empirical data set for finetuning the convolutional neural network following the training comprises:
generating a plurality of geometric objects to be placed inside a background image for capturing by the imaging apparatus; and
receiving, from the imaging apparatus, a plurality of finetuning images based on the plurality of geometric objects placed inside the background image, the plurality of finetuning images defining the empirical data set.
5. The system of claim 4 , wherein the operations further comprise:
introducing shape irregularities to the plurality of geometric objects placed inside the background image.
6. The system of claim 1 , wherein the convolutional neural network comprises a downsampling portion followed by an upsampling portion.
7. The system of claim 1 , wherein the computing system is a component of the imaging apparatus.
8. A method comprising:
obtaining, by a computing system, a plurality of noisy images of a specimen captured by an imaging apparatus using darkfield imaging, wherein the plurality of noisy images includes at least two images of the specimen captured using darkfield imaging;
denoising, by the computing system, the plurality of noisy images by inputting the plurality of noisy images of the specimen into a convolutional neural network trained to output a single denoised image of the specimen; and
generating, as output from the convolutional neural network, the single denoised image of the specimen.
9. The method of claim 8 , further comprising:
generating, by the computing system, a synthetic data set for training the convolutional neural network, the synthetic data set comprising a plurality of synthetic images and, for each synthetic image, a plurality of noisy images derived from the respective synthetic image; and
training, by the computing system, the convolutional neural network to denoise a target plurality of noisy images based on the synthetic data set.
10. The method of claim 9 , further comprising:
generating, by the computing system, an empirical data set for finetuning the convolutional neural network following the training; and
finetuning, by the computing system, the convolutional neural network based on the empirical data set.
11. The method of claim 10 , wherein generating, by the computing system, the empirical data set for finetuning the convolutional neural network following the training comprises:
generating a plurality of geometric objects to be placed inside a background image for capturing by the imaging apparatus; and
receiving, from the imaging apparatus, a plurality of finetuning images based on the plurality of geometric objects placed inside the background image, the plurality of finetuning images defining the empirical data set.
12. The method of claim 11 , further comprising:
introducing shape irregularities to the plurality of geometric objects placed inside the background image.
13. The method of claim 8 , wherein the convolutional neural network comprises a downsampling portion followed by an upsampling portion.
14. The method of claim 8 , wherein the computing system is a component of the imaging apparatus.
15. A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by one or more processors, causes a computing system to perform operations, comprising:
obtaining, by the computing system, a plurality of noisy images of a specimen captured by an imaging apparatus using darkfield imaging, wherein the plurality of noisy images includes at least two images of the specimen captured using darkfield imaging;
denoising, by the computing system, the plurality of noisy images by inputting the plurality of noisy images of the specimen into a convolutional neural network trained to output a single denoised image of the specimen; and
generating, as output from the convolutional neural network, the single denoised image of the specimen.
16. The non-transitory computer readable medium of claim 15 , further comprising:
generating, by the computing system, a synthetic data set for training the convolutional neural network, the synthetic data set comprising a plurality of synthetic images and, for each synthetic image, a plurality of noisy images derived from the respective synthetic image; and
training, by the computing system, the convolutional neural network to denoise a target plurality of noisy images based on the synthetic data set.
17. The non-transitory computer readable medium of claim 16 , further comprising:
generating, by the computing system, an empirical data set for finetuning the convolutional neural network following the training; and
finetuning, by the computing system, the convolutional neural network based on the empirical data set.
18. The non-transitory computer readable medium of claim 17 , wherein generating, by the computing system, the empirical data set for finetuning the convolutional neural network following the training comprises:
generating a plurality of geometric objects to be placed inside a background image for capturing by the imaging apparatus; and
receiving, from the imaging apparatus, a plurality of finetuning images based on the plurality of geometric objects placed inside the background image, the plurality of finetuning images defining the empirical data set.
19. The non-transitory computer readable medium of claim 18 , further comprising:
introducing shape irregularities to the plurality of geometric objects placed inside the background image.
20. The non-transitory computer readable medium of claim 15 , wherein the convolutional neural network comprises a downsampling portion followed by an upsampling portion.