SYSTEM AND METHOD FOR REAL-TIME VARIABLE RESOLUTION MICROSCOPE SLIDE IMAGING
The present invention provides a system and method for capturing images during review of a microscope slide. In certain embodiments, such system and method allow for the capture of images and construction of a composited microscope mosaic image within the workflow of the slide reviewer, such as a pathologist reviewing a tissue sample. In certain embodiments, said mosaic images are whole slide images constructed by and capable of being viewed at variable resolutions and magnifications corresponding to the review of the original microscope slide by the slide reviewer.
1 . A method for capturing images during review of a microscope slide using a microscope adapted to connection to a digital camera comprises steps of:
(a) receiving, by a controller from the digital camera, first image information, wherein the controller is operably connected to the digital camera and the microscope,
wherein the microscope includes a stage configured to support the microscope slide, a first objective lens providing a first magnification, a second objective lens providing a second magnification, a third objective lens providing a third magnification and a lens indicator sensor,
wherein the microscope slide is movable relative to the first objective lens, the second objective lens and the third objective lens,
wherein the lens indicator sensor is configured to provide lens information indicating whether the first objective lens, the second objective lens or the third objective lens is in use and including a first time stamp,
wherein the first image information is associated with images provided via the first objective lens, the second objective lens and the third objective lens during review of the microscope slide,
wherein the first image information includes:
(i) image data; and
(ii) a second time stamp associated with a time at which such image data was generated by the digital camera;
(b) receiving, by the controller from the microscope, the lens information;
(c) receiving, by the controller from the digital camera, the first image information;
(d) capturing, by the controller, a plurality of digital images based on the first image information and the second time stamp, wherein a first digital image of the plurality of digital images is associated with the first objective lens having the first magnification and a second digital image of the plurality of digital images is associated with the second objective lens having the second magnification;
(e) storing, by the controller, the plurality of digital images and the lens information in a memory;
(f) selecting, by the controller, a first set of digital images from the plurality of digital images for inclusion in a mosaic image, wherein the first set of digital images
includes at least a first digital image associated with the first objective lens, at least a second digital image associated with the second objective lens and at least
a third digital image associated with the third objective lens,
wherein the selecting is based on the first image information and the lens information;
(g) generating, by the controller, a mosaic image using the first set of digital images and including at least the first digital image, the second digital image and the third digital image so that the mosaic image includes digital images of the first magnification, second magnification and third magnification,
wherein the generating includes:
matching at least the first digital image and at least one of the second digital image and the third digital image using a matching algorithm; and
(h) storing, by the controller, the first set of digital images in the memory.
2 . The method of claim 1 , wherein a pixel size of the digital camera is two times smaller than a diffraction limited spot size d provided by the first objective lens, second objective lens and third objective lens wherein:
d
=
1
.
2
2
λ
(
NA
obj
+
NA
cond
)
where λ is the wavelength of light, NA obj is a numerical aperture of a selected objective lens, and NA cond is the numerical aperture of a condenser lens.
3 . The method of claim 1 , wherein a third digital image of the plurality of digital images is associated with the third objective lens having the third magnification.
4 . A method for capturing images during review of a microscope slide using a microscope adapted to connection to a digital camera comprises steps of:
(a) receiving, by a controller from the digital camera, first image information, wherein the controller is operably connected to the digital camera and the microscope,
wherein the microscope includes a stage configured to support the microscope slide, a first objective lens providing a first magnification, a second objective lens providing a second magnification, a third objective lens providing a third magnification and a lens indicator sensor,
wherein the microscope slide is movable relative to the first objective lens, the second objective lens and the third objective lens,
wherein the lens indicator sensor is configured to provide lens information indicating whether the first objective lens, the second objective lens or the third objective lens is in use and including a first time stamp,
wherein the first image information is associated with images provided via the first objective lens, the second objective lens and the third objective lens during review of the microscope slide,
wherein the first image information includes:
(i) image data; and
(ii) a second time stamp associated with a time at which such image data was generated by the digital camera;
(b) receiving, by the controller from the microscope, the lens information;
(c) receiving, by the controller from the digital camera, the first image information;
(d) capturing, by the controller, a plurality of digital images based on the first image information and the second time stamp;
(e) storing, by the controller, the plurality of digital images and the lens information in a memory;
(f) selecting, by the controller, a first set of digital images from the plurality of digital images for inclusion in a mosaic image, wherein the first set of digital images
includes at least a first digital image associated with the first objective lens, at least a second digital image associated with the second objective lens and at least
a third digital image associated with the third objective lens,
wherein the selecting is based on the first image information and the lens information;
(g) generating, by the controller, a mosaic image using the first set of digital images and including at least the first digital image, the second digital image and the third digital image so that the mosaic image includes digital images of the first magnification, second magnification and third magnification,
wherein the generating includes:
matching at least the first digital image and at least one of the second digital image and the third digital image using a matching algorithm; and
(h) storing, by the controller, the first set of digital images in the memory;
(i) generating, by the controller, at least one resolution map indicating for each pixel in a set of pixels associated with at least one respective objective lens selected from the group comprising the first objective lens, the second objective lens and the third objective lens;
(j) generating, by the controller, at least one dwell time map indicating a duration of time each image of the first set of digital images included in the mosaic image was viewed;
(k) generating, by the controller, a path map indicating a respective path of each respective objective lens during the review of the microscope slide.
5 . The method of claim 4 , wherein a pixel size of the digital camera is two times smaller than a diffraction limited spot size d provided by the first objective lens, second objective lens and third objective lens wherein:
d
=
1
.
2
2
λ
(
NA
obj
+
NA
cond
)
where λ is the wavelength of light, NA obj is a numerical aperture of a selected objective lens, and NA cond is the numerical aperture of a condenser lens.
6 . A method for capturing images during review of a microscope slide using a microscope adapted to connection to a digital camera comprises steps of:
(a) receiving, by a controller from the digital camera, first image information, wherein the controller is operably connected to the digital camera and the microscope,
wherein the microscope includes a stage configured to support the microscope slide, a first objective lens providing a first magnification, a second objective lens providing a second magnification, a third objective lens providing a third magnification and a lens indicator sensor,
wherein the microscope slide is movable relative to the first objective lens, the second objective lens and the third objective lens,
wherein the lens indicator sensor is configured to provide lens information indicating whether the first objective lens, the second objective lens or the third objective lens is in use and including a first time stamp,
wherein the first image information is associated with images provided via the first objective lens, the second objective lens and the third objective lens during review of the microscope slide,
wherein the first image information includes:
(i) image data; and
(ii) a second time stamp associated with a time at which such image data was generated by the digital camera;
(b) receiving, by the controller from the microscope, the lens information;
(c) receiving, by the controller from the digital camera, the first image information;
(d) capturing, by the controller, a plurality of digital images based on the first image information and the second time stamp;
(e) storing, by the controller, the plurality of digital images and the lens information in a memory;
(f) selecting, by the controller, a first set of digital images from the plurality of digital images for inclusion in a mosaic image, wherein the first set of digital images
includes at least a first digital image associated with the first objective lens, at least a second digital image associated with the second objective lens and at least
a third digital image associated with the third objective lens,
wherein the selecting is based on the first image information and the lens information;
(g) generating, by the controller, a mosaic image using the first set of digital images and including at least the first digital image, the second digital image and the third digital image so that the mosaic image includes digital images of the first magnification, second magnification and third magnification,
wherein the generating includes:
matching at least the first digital image and at least one of the second digital image and the third digital image using a matching algorithm; and
(h) storing, by the controller, the first set of digital images in the memory
(i) generating, by the controller, path information indicating a path of the first objective lens, second objective lens and third objective lens across the microscope slide during the review, wherein the path information includes:
i. position data;
ii. lens data; and
iii. timing information.
7 . The method of claim 6 , wherein a pixel size of the digital camera is two times smaller than a diffraction limited spot size d provided by the first objective lens, second objective lens and third objective lens wherein:
d
=
1
.
2
2
λ
(
NA
obj
+
NA
cond
)
where λ is the wavelength of light, NA obj is a numerical aperture of a selected objective lens, and NA cond is the numerical aperture of a condenser lens.
8 . The method of claim 6 , wherein the path information is based on the plurality of digital images.
9 . The method of claim 6 , wherein the path information is stored in the memory.
10 . The method of claim 6 , wherein the path information includes magnification information associated with each position, wherein the magnification information is based on the lens information.
11 . The method of claim 6 , further comprising a step of generating, by the controller, a path map indicating the path of the first objective lens, second objective lens and third objective lens across the microscope slide during the review based on the path information.
12 . The method of claim 11 , wherein the step of generating the path map includes:
i. choosing a coordinate on each digital image and localizing that coordinate within the mosaic image;
ii. determining a time sequence and velocity of the microscope slide based on a respective third time stamp associated with each of the coordinates.