Method, device, and computer program for improving the reconstruction of dense super-resolution images from diffraction-limited images acquired by single molecule localization microscopy
The invention relates to reconstructing a synthetic dense super-resolution image from at least one low-information-content image, for example from a sequence of diffraction-limited images acquired by single molecule localization microscopy. After having obtained such a sequence of diffraction-limited images, a sparse localization image is reconstructed from the obtained sequence of diffraction-limited images according to single molecule localization microscopy image processing. The reconstructed sparse localization image and/or a corresponding low-resolution wide-field image are input to an artificial neural network and a synthetic dense super-resolution image is obtained from the artificial neural network, the latter being trained with training data comprising triplets of sparse localization images, at least partially corresponding low-resolution wide-field images, and corresponding dense super-resolution images, as a function of a training objective function comparing dense super-resolution images and corresponding outputs of the artificial neural network.
1. A computer method for reconstructing at least one synthetic dense super-resolution image (ANNA-SMLM(ki)) from at least one low-information-content image (SMLM(ki), Li), the at least one low-information-content image comprising a low-resolution wide-field image (Li), the method comprising:
obtaining the at least one low-information-content image;
inputting the at least one obtained low-information-content image to an artificial neural network, and
obtaining the synthetic dense super-resolution image from the artificial neural network,
wherein the artificial neural network has been trained with training data comprising triplets of sparse localization images (SMLM(ki)) obtained from sequences of diffraction-limited images acquired by single molecule localization microscopy, at least partially corresponding low-resolution wide-field images (Li), and corresponding dense super-resolution images (SMLM(Ki)), as a function of a training objective function comparing dense super-resolution images and corresponding outputs of the artificial neural network.
2. The method according to claim 1 , wherein the at least one low-information-content image further comprises a sparse localization image (SMLM(ki)) reconstructed from a sequence of diffraction-limited images according to single molecule localization microscopy image processing.
3. The method according to claim 2 , further comprising:
obtaining a sequence of diffraction-limited images; and
reconstructing the sparse localization image from the obtained sequence of diffraction-limited images.
4. The method according to claim 3 , wherein the low-resolution wide-field image corresponds at least partially to the diffraction-limited images used for reconstructing the sparse localization image.
5. The method according to claim 4 , further comprising computing an error map, the error map representing a probability of errors to occur as a function of locations in the synthetic dense super-resolution image.
6. The method according to claim 5 , wherein the computing of the error map comprises generating a low-resolution image (L′(k)) from the synthetic dense super-resolution image, the generated low-resolution image being compared with the low-resolution wide-field image inputted into the artificial neural network.
7. The method according to claim 1 , wherein the training objective function comprises at least one parameter to select a subset of the training data to be inputted into the artificial neural network.
8. The method according to claim 1 , wherein the training objective function comprises at least one of an objective function based on an artificial neural network of the conditional generative adversarial network type and wherein the low-resolution wide-field image inputted into the artificial neural network is used as a condition of the conditional generative adversarial network.
9. The method according to claim 1 , wherein the sparse localization images are 3D sparse localization images and the dense super-resolution images are 3D dense super-resolution images.
10. The method according to claim 1 , wherein the sparse localization images are multicolor sparse localization images and the dense super-resolution images are multicolor dense super-resolution images.
11. A computer method for reconstructing at least one synthetic dense super-resolution image (ANNA-SMLM′(ki)) from a sequence of diffraction-limited images acquired by single molecule localization microscopy, the method comprising:
obtaining the sequence of diffraction-limited images acquired by single molecule localization microscopy;
reconstructing a sparse localization image (SMLM′(ki)) from the obtained sequence of diffraction-limited images according to single molecule localization microscopy image processing;
inputting the reconstructed sparse localization image to an artificial neural network, and
obtaining the synthetic dense super-resolution image from the artificial neural network,
wherein the artificial neural network has been trained with training data comprising pairs of sparse localization images (SMLM(ki)) and corresponding dense super-resolution images (SMLM(Ki)), as a function of a training objective function comparing dense super-resolution images and corresponding outputs of the artificial neural network.
12. The method according to claim 11 , wherein the diffraction-limited images of the obtained sequence correspond to a first field of view of a field of study, and the obtaining of the sequence of diffraction-limited images, the reconstructing of the sparse localization image from the obtained sequence of diffraction-limited images, the inputting of the reconstructed sparse localization image to the artificial neural network, and the obtaining of the synthetic dense super-resolution image from the artificial neural network being repeated for at least one second field of view of the field of study, the first and the second fields of view being different.
13. The method according to claim 12 , wherein the first field of view and the second field of view are contiguous within the field of study, the method further comprising creating a resulting image by stitching synthetic dense super-resolution images obtained from the artificial neural network.
14. The method according to claim 11 , wherein the diffraction-limited images of the obtained sequence correspond to a predetermined field of view, and the obtaining of the sequence of diffraction-limited images, the reconstructing of the sparse localization image from the obtained sequence of diffraction-limited images, the inputting of the reconstructed sparse localization image to the artificial neural network, and the obtaining of the synthetic dense super-resolution image from the artificial neural network being repeated periodically for the same field of view.
15. The method according to claim 11 , further comprising training the artificial neural network and pairing sparse localization images and corresponding dense super-resolution images for training the artificial neural network.
16. The method according to claim 15 , wherein the pairing of the sparse localization images and corresponding dense super-resolution images comprises:
obtaining a training sequence of diffraction-limited images acquired by single molecule localization microscopy;
reconstructing a plurality of sparse localization images from first sub-sets of the obtained training sequence of diffraction-limited images according to a single molecule localization microscopy image processing; and
reconstructing a plurality of dense super-resolution images from second sub-sets of the obtained training sequence of diffraction-limited images according to a single molecule localization microscopy image processing, each of the second sub-sets comprising more diffraction-limited images than each of the first sub-sets.
17. The method according to claim 11 , wherein the sparse localization images are 3D sparse localization images and the dense super-resolution images are 3D dense super-resolution images.
18. The method according to claim 11 , wherein the sparse localization images are multicolor sparse localization images and the dense super-resolution images are multicolor dense super-resolution images.
19. An apparatus for reconstructing at least one synthetic dense super-resolution image (ANNA-SMLM(ki)) from at least one low-information content image (SMLM(ki), Li), the at least one low-information-content image comprising a low-resolution wide-field image (Li), the apparatus being configured to carry out the following method:
obtaining the at least one low-information-content image;
inputting the at least one obtained low-information-content image to an artificial neural network; and
obtaining the synthetic dense super-resolution image from the artificial neural network;
wherein the artificial neural network has been trained with training data comprising triplets of sparse localization images (SMLM(ki)) obtained from sequences of diffraction-limited images acquired by single molecule localization microscopy, at least partially corresponding low-resolution wide-field images (Li), and corresponding dense superresolution images (SMLM(Ki)), as a function of a training objective function comparing dense super-resolution images and corresponding outputs of the artificial neural network.
20. A non-transitory computer storage medium for a programmable apparatus, the non-transitory computer storage medium storing instructions for carrying out the method according to claim 1 when the program is loaded and executed by a programmable apparatus.