Systems and methods for camera-based image processing in microscopy instruments
View Patent ↗Systems and methods for executing super-resolution microscopy of a specimen with most of the image processing performed in a camera of a fluorescence microscopy instrument are described. In one aspect, the camera includes one or more processors to execute machine-readable instructions that control excitation light output from a multi-channel light source, control capture of intermediate images of the specimen, and perform image processing of the intermediate images to produce a final super-resolution image of the specimen.
1. A camera comprising:
one or more processors; and
a computer-readable medium having machine-readable instructions recorded thereon,
wherein the one or more processors is configured to execute the machine-readable instructions for:
capturing light emitted from a subset of light emitters within a specimen exposed to a light source, to record an unprocessed intermediate image of the subset of light emitters in the camera;
processing the recorded unprocessed intermediate image and creating a processed intermediate image of the subset of light emitters, wherein the processed intermediate image comprises a plurality of spots, each spot associated with an individual image of a light emitter of the subset of light emitters; and
obtaining, for each individual image, a centroid coordinate of said each individual image.
2. The camera of claim 1 , wherein the recorded unprocessed intermediate image is a diffraction-limited image.
3. The camera of claim 1 , wherein each light emitter of the subset of light emitters is separated from one another by a distance greater than a diffraction-limited distance.
4. The camera of claim 3 , wherein the diffraction-limited distance is approximately 200 nm.
5. The camera of claim 1 , wherein the centroid coordinate is determined to a resolution of below 10 nm.
6. The camera of claim 1 , wherein using the camera to obtain the centroid coordinate associated with said each individual image further comprises using the camera to curve fit a Gaussian distribution to said each individual image.
7. The camera of claim 6 , wherein the centroid coordinate associated with said each individual image is a (x, y) coordinate associated with a maximum of the Gaussian distribution to said each individual image.
8. The camera of claim 6 , wherein an intensity value is assigned to the centroid coordinate, and the intensity value associated with the centroid coordinate forms a pixel in the processed intermediate image.
9. The camera of claim 1 , wherein the specimen is fluorophore labeled.
10. The camera of claim 1 , wherein the instructions further comprise storing the processed intermediate image in the camera.
11. The camera of claim 1 , wherein the instructions further comprise transferring the processed intermediate image and each centroid coordinate of its corresponding individual image to a computer in communication with the camera, wherein the computer is configured to produce a final image of the specimen.
12. The camera of claim 11 , wherein the final image is a super-resolution image.
13. The camera of claim 1 , wherein the instructions further comprise:
capturing light emitted from a plurality of subsequent subsets of light emitters within the specimen to record a plurality of subsequent unprocessed intermediate images in the camera,
processing the recorded subsequent images and creating a plurality of subsequent processed intermediate images, wherein each of the plurality of the subsequent processed images comprises a plurality of individual spots, each spot corresponding to an individual image of a light emitter of subsequent subsets of light emitters, wherein at least one of the plurality of the subsequent subsets of light emitters is a different subset of light emitters.
14. The camera of claim 13 , wherein the instructions further comprise transferring, the processed intermediate image and the plurality of subsequent processed intermediate images, and each centroid coordinate of its corresponding individual image, to a computer in communication with the camera, wherein the computer is configured to store and combine the processed intermediate image with the subsequent processed images to produce the final image.
15. The camera of claim 1 , wherein the one or more processors comprises a digital signal processor to process the recorded intermediate image.
16. The camera of claim 1 , wherein the one or more processors comprises a graphics processing unit to process the recorded intermediate image.
17. The camera of claim 1 , wherein the one or more processors comprises an application-specific integrated circuit to process the recorded intermediate image.
18. A method for capturing and processing images of a specimen using a camera, the method comprising:
capturing light emitted from a subset of light emitters within the specimen being exposed to a light source, to record an unprocessed intermediate image of the subset of light emitters in the camera;
using the camera to process the recorded unprocessed intermediate image and create a processed intermediate image of the subset of light emitters, wherein the processed intermediate image comprises a plurality of spots, each spot associated with an individual image of a light emitter of the subset of light emitters;
using the camera to obtain, for each individual image, a centroid coordinate of said each individual image.
19. The method of claim 18 , further comprising:
transferring, the processed intermediate image and each centroid coordinate of its corresponding individual image, to a computer in communication with the camera, wherein the computer is configured to produce a final image of the specimen.
20. The method of claim 19 , wherein the final image is a super-resolution image.