Accelerating digital microscopy scans using empty/dirty area detection
A microscope including an illumination assembly, an image capture device and a processor can be configured to selectively identify regions of a sample including artifacts or empty space. By selectively identifying regions of the sample that have artifacts or empty space, the amount of time to generate an image of the sample and resources used to generate the image can be decreased substantially while providing high resolution for appropriate regions of the computational image. The processor can be configured to change the imaging process in response to regions of the sample that includes artifacts or empty space. The imaging process may include a higher resolution process to output higher resolution portions of the computational image for sample regions including valid sample material, and a lower resolution process to output lower resolution portions of the computational image for sample regions including valid sample material.
1. A microscope, comprising:
an illumination assembly operable to illuminate a sample under observation of the microscope;
an image capture device operable to capture an initial image set of the illuminated sample; and
a processor coupled to the image capture device and configured with instructions to identify an area of the sample that comprises at least one of artifact or empty space, and to determine a process for generating a computational image of the sample in response to identifying the area;
wherein the processor is configured to output to a display portions of the computational image of the sample that comprise the at least one of artifact or empty space at a higher rate than other portions of the computational image of the sample that do not comprise the at least one of artifact or empty space.
2. The microscope of claim 1 , wherein:
the processor is configured with instructions to operate in a first mode in response to identifying the area of the sample comprising the at least one of artifact or empty space and in a second mode in response to identifying an area of the sample that does not comprise the at least one of artifact or empty space; and
in the first mode and the second mode the processor is configured to output portions of the computational image with a first spatial resolving power and second spatial resolving power, respectively, the first spatial resolving power less than the second spatial resolving power.
3. The microscope of claim 1 , wherein:
the image set comprises a plurality of regions;
the processor is configured with instructions to run a first process to increase spatial resolving power of each of the plurality of regions and to run a second process to determine whether each of the plurality of regions comprises the at least one of artifact or empty space;
optionally the second process is initiated before the first process has completed; and
optionally the first process is stopped in response to the second process identifying one or more of the plurality of regions comprising the at least one of artifact or empty space.
4. The microscope of claim 1 , wherein:
the illumination assembly is configured to illuminate the sample at a plurality of illumination conditions and the image capture device generates a plurality of images, each of the plurality of images corresponding to a different illumination condition; and
the computational image is generated from the plurality of images;
the at least one of artifact or empty space is identified from the computational image or one or more of the plurality of images; and
the plurality of illumination conditions comprises at least one of an illumination angle, an illumination wavelength, or an illumination pattern.
5. The microscope of claim 1 , wherein:
the processor is configured with instructions to detect the area of the sample that comprises the at least one of artifact or empty space, by using one or more of: a classifier, a detail in a darkfield image, a brightness, a color distribution, a cross correlation, a Fourier analysis, an entropy, a feature detection, a shift of details, a sparsity, an edge detection, a three-dimensional reconstruction, a depth detection, or a light field.
6. The microscope of claim 1 , further comprising:
an input comprising pixels from an image set of the sample, and an output comprising the at least one of artifact or empty space.
7. The microscope of claim 1 , wherein:
the computational image comprises a two-dimensional image.
8. The microscope of claim 1 , wherein:
the computational image comprises one or more of a three-dimensional image or a three-dimensional model of the sample.
9. The microscope of claim 1 , further comprising:
another image capture device with a higher resolving power than said image capture device, the another image capture device being used to image the sample at the higher resolving power than said image capture device.
10. The microscope of claim 1 , wherein:
the initial image set comprises a preview image; and
the processor is configured with instructions to determine a boundary of the sample in the preview image, and to generate another image of the sample, wherein the another image comprises a higher resolving power within the boundary of the sample and a lower resolving power in the identified area.
11. The microscope of claim 10 , wherein:
the higher resolving power corresponds to smaller spatial features being resolved than the lower resolving power.
12. The microscope of claim 1 , wherein:
the computational image comprises a digital image; and
the processor is configured with instructions to remove data pertaining to the identified area to change a resolving power of the identified area.
13. The microscope of claim 1 , wherein:
the processor is configured with instructions to identify the area as comprising the at least one of artifact or empty space in response to a shift in position of portions of the initial image set or the computational image in response to an off-axis illumination of the sample.
14. The microscope of claim 1 , wherein:
the microscope comprises a computational microscope configured to generate an image from a plurality of images of the sample with different illumination conditions; and
the different illumination conditions comprise at least one of different illumination angles, different illumination wavelengths, different illumination patterns, different illumination durations, different illumination intensities, or different illumination positions.
15. The microscope of claim 1 , further comprising:
a translation stage to move the sample to a plurality of locations in order to scan the sample and generate a plurality of images, wherein the microscope is configured to scan the sample faster in response to the area that comprises the at least one of artifact or empty space as compared to a scan speed of an area of the sample that comprises a valid portion of the sample.
16. The microscope of claim 1 , wherein:
the image capture device captures the initial image set of the illuminated sample using a plurality of illumination conditions for illuminating the sample; and
the plurality of illumination conditions comprises at least one of different illumination angles, different illumination wavelengths, different illumination patterns, different illumination durations, different illumination intensities, or different illumination positions.
17. The microscope of claim 1 , wherein:
the processor is configured with instructions to reduce computational complexity when processing in the area of the sample that comprises the at least one of artifact or empty space to reduce computation time.
18. The microscope of claim 1 , wherein:
the processor is configured with instructions to use fewer illumination conditions when processing in the area of the sample that comprises the at least one of artifact or empty space as compared to an area of the sample that does not comprise the at least one of artifact or empty space.
19. The microscope of claim 1 , wherein:
the artifact comprises one or more of particulate matter, debris, dirt, dust, or smudges.