Volumetric multi-modal microscopy methods and systems
Methods and apparatus for obtaining 3D imaging of tissue involve scanning a focused laser beam in xz planes to obtain a set of xz plane images spaced apart in a y direction. The xz plane images are processed to correct distortions and motions and combined to provide 3D image data. Surface flatting is optionally performed. Imaging may be performed using a femtosecond (fs) laser beam. Different components of light returning from the tissue may be detected and processed to yield plural co-registered images using different imaging modalities, for example, reflective confocal microscopy (RCM), two photon fluorescence (TPF) and second harmonic generation (SHG).
1. A method for imaging tissue, the method comprising:
focusing a beam of light to a focus point;
raster scanning the location of the focus point across a field of view in a vertical plane extending generally perpendicular to a surface of the tissue and collecting and detecting light returning from the focus point to provide a first xz plane image;
scanning the location of the focus point in a y-direction transverse to the vertical plane to obtain a sequence of additional xz plane images corresponding to vertical planes parallel to and spaced apart in a y-direction from the vertical plane, the first xz plane image and the additional xz plane images constituting a set of xz plane images; and
processing the set of xz plane images to provide volumetric image data for the tissue;
wherein scanning the location of the focus point in the y direction comprises holding the objective fixed in the y direction while moving the tissue in the y direction.
2. The method according to claim 1 wherein focusing the beam of light comprises passing the light through an objective and raster scanning the location of the focus point comprises optically scanning the beam in an x direction in coordination with scanning the objective relative to the tissue along a z axis which is generally perpendicular to the x direction and to the surface of the tissue.
3. The method according to claim 1 wherein moving the tissue in the y direction comprises operating a linear motion stage coupled between the objective and a member adhered to a surface of the tissue.
4. The method according to claim 1 wherein a range of the scanning in the y direction is at least three times larger than a range of the scanning in the x direction.
5. The method according to claim 1 comprising separating and detecting a plurality of components of the light returning from the focus point using a plurality of different light detectors and processing output signals from the plurality of light detectors to provide a plurality of co-registered images according to plural imaging modalities.
6. A method for imaging tissue, the method comprising:
focusing a beam of light to a focus point;
raster scanning the location of the focus point across a field of view in a vertical plane extending generally perpendicular to a surface of the tissue and collecting and detecting light returning from the focus point to provide a first xz plane image;
scanning the location of the focus point in a y-direction transverse to the vertical plane to obtain a sequence of additional xz plane images corresponding to vertical planes parallel to and spaced apart in a y-direction from the vertical plane, the first xz plane image and the additional xz plane images constituting a set of xz plane images; and
processing the set of xz plane images to provide volumetric image data for the tissue;
wherein focusing the beam of light comprises passing the light through an objective and raster scanning the location of the focus point comprises optically scanning the beam in an x direction in coordination with scanning the objective relative to the tissue along a z axis which is generally perpendicular to the x direction and to the surface of the tissue;
wherein processing the set of xz plane image frames comprises performing distortion compensation, the distortion compensation comprising monitoring an output of a z-axis position sensor to determine an actual trajectory of the objective during the scanning along the z axis and shifting pixels of the xz image according to the actual trajectory.
7. The method according to claim 6 wherein the scanning along the z axis is bidirectional, and the xz plane images include first xz plane images acquired while scanning the objective in a first direction along the z axis and second xz plane images acquired while scanning the objective in a second direction opposed to the first direction along the z axis.
8. The method according to claim 7 comprising monitoring the output of the z-axis position sensor to determine a first actual trajectory of the objective during the scanning along the z axis in the first direction and a second actual trajectory of the objective during the scanning along the z axis in the second direction and shifting the positions of the pixels of the first xz plane images in the z direction based on the first actual trajectory and shifting the pixels of the second xz plane images in the z direction based on the second actual trajectory.
9. A method for imaging tissue, the method comprising:
focusing a beam of light to a focus point;
raster scanning the location of the focus point across a field of view in a vertical plane extending generally perpendicular to a surface of the tissue and collecting and detecting light returning from the focus point to provide a first xz plane image;
scanning the location of the focus point in a y-direction transverse to the vertical plane to obtain a sequence of additional xz plane images corresponding to vertical planes parallel to and spaced apart in a y-direction from the vertical plane, the first xz plane image and the additional xz plane images constituting a set of xz plane images; and
processing the set of xz plane images to provide volumetric image data for the tissue;
wherein processing the set of xz plane image frames comprises performing motion compensation, the motion compensation comprising processing the xz plane images to determine representative z-axis locations of the tissue surface in each of the xz plane images and shifting the pixels of the xz plane images based on the representative z-axis locations.
10. The method according to claim 9 wherein shifting the pixels of the xz plane images based on the representative z-axis locations comprises determining differences between the representative z-axis locations for adjacent ones of a sequence of the xz plane images, for each of the xz plane images in the sequence calculating a cumulative difference of the representative z-axis locations relative to a selected one of the xz plane images and shifting the pixels of each of the xz plane images in the sequence by the corresponding cumulative difference.
11. The method according to claim 9 wherein the first xz plane image and the additional xz plane images constitute a first set of xz plane images for a first imaging modality and the method comprises acquiring one or more additional sets of xz plane images, each of the additional sets of xz plane images for a corresponding additional imaging modality and co-registered with the first set of xz plane images and the method comprises shifting the pixels of the xz plane images of the additional sets of xz plane images based on the representative z-axis locations determined for the first set of xz plane images.
12. A method for imaging tissue, the method comprising:
focusing a beam of light to a focus point;
raster scanning the location of the focus point across a field of view in a vertical plane extending generally perpendicular to a surface of the tissue and collecting and detecting light returning from the focus point to provide a first xz plane image;
scanning the location of the focus point in a y-direction transverse to the vertical plane to obtain a sequence of additional xz plane images corresponding to vertical planes parallel to and spaced apart in a y-direction from the vertical plane, the first xz plane image and the additional xz plane images constituting a set of xz plane images; and
processing the set of xz plane images to provide volumetric image data for the tissue;
wherein processing the set of xz plane image frames comprises performing surface flatting, wherein the surface flatting comprises:
processing the xz plane images to yield a 3D profile of the tissue surface;
smoothing the 3D profile of the tissue surface;
determining offsets corresponding to columns in the xz plane images that extend parallel to the z axis based on the smoothed 3D profile of the tissue surface; and
applying the offsets to the xz plane images.
13. The method according to claim 12 wherein the first xz plane image and the additional xz plane images constitute a first set of xz plane images for a first imaging modality and the method comprises acquiring one or more additional sets of xz plane images, each of the additional sets of xz plane images for a corresponding additional imaging modality and co-registered with the first set of xz plane images and the method comprises applying the offsets to the xz plane images of the additional sets of xz plane images.
14. Apparatus for imaging tissue, the apparatus comprising:
a laser light source operative to emit a beam of light;
an objective arranged to receive and focus the beam of light to a focus point;
a scanner unit in an optical path between the laser light source and the objective, the scanner unit operative to redirect the beam of light and thereby cause the focus point to scan in an x direction;
a z-axis scanner operative to move the objective in a z direction parallel to an optical axis of the objective,
a first light detector arranged to receive light returning through the objective from the focus point and to generate an output signal representing an intensity of the light returning,
a tissue stabilizing mechanism coupled to the objective by way of a movable stage operative to move tissue relative to the objective in a y direction that is transverse relative to the x direction while holding the objective fixed,
a controller configured to generate coordinated driving signals for the scanner unit and the z-axis scanner to cause the focus point to be raster scanned in an xz plane and control signals for the movable stage to cause the focus point to be scanned in the y direction,
a frame grabber operative to monitor the output signal in coordination with the raster scanning of the focus point to build a first series of xz image frames, and
a data processor connected to receive the xz image frames and configured to process the xz image frames to provide volumetric image data for a volume scanned by the focus point.
15. The apparatus according to claim 14 comprising:
a first wavelength selective element in the optical path between the scanning unit and the objective, the first wavelength selective element arranged to redirect a portion of the returning light having a wavelength that is shorter than a wavelength of the beam of light out of the beam and a second light detector arranged to detect light from the portion of the returning light; and
a second wavelength selective element arranged to receive the portion of the returning light redirected by the first wavelength selective element and to split the portion of the returning light into first and second parts; and
a third light detector arranged to detect light of the third part
wherein the second light detector is arranged to detect light of the second part.
16. The apparatus according to claim 14 wherein a range of scanning in the y direction is at least three times larger than a range of scanning in the x direction.
17. Apparatus for imaging tissue, the apparatus comprising:
a laser light source operative to emit a beam of light;
an objective arranged to receive and focus the beam of light to a focus point;
a scanner unit in an optical path between the laser light source and the objective, the scanner unit operative to redirect the beam of light and thereby cause the focus point to scan in an x direction;
a z-axis scanner operative to move the objective in a z direction parallel to an optical axis of the objective,
a first light detector arranged to receive light returning through the objective from the focus point and to generate an output signal representing an intensity of the light returning,
a tissue stabilizing mechanism coupled to the objective by way of a movable stage operative to move tissue relative to the objective in a y direction that is transverse relative to the x direction,
a controller configured to generate coordinated driving signals for the scanner unit and the z-axis scanner to cause the focus point to be raster scanned in an xz plane and control signals for the movable stage to cause the focus point to be scanned in the y direction,
a frame grabber operative to monitor the output signal in coordination with the raster scanning of the focus point to build a first series of xz image frames, and
a data processor connected to receive the xz image frames and configured to process the xz image frames to provide volumetric image data for a volume scanned by the focus point;
a first wavelength selective element in the optical path between the scanning unit and the objective, the first wavelength selective element arranged to redirect a portion of the returning light having a wavelength that is shorter than a wavelength of the beam of light out of the beam and a second light detector arranged to detect light from the portion of the returning light; and
a second wavelength selective element arranged to receive the portion of the returning light redirected by the first wavelength selective element and to split the portion of the returning light into first and second parts; and
a third light detector arranged to detect light of the third part
wherein the second light detector is arranged to detect light of the second part;
wherein an output of the third light detector is connected to the frame grabber and the frame grabber is configured to build a third series of xz image frames co-registered with the first series of xz image frames and an output of the second light detector is connected to the frame grabber and the frame grabber is configured to build a second series of xz image frames co-registered with the first series of xz image frames.
18. Apparatus for imaging tissue, the apparatus comprising:
a laser light source operative to emit a beam of light;
an objective arranged to receive and focus the beam of light to a focus point;
a scanner unit in an optical path between the laser light source and the objective, the scanner unit operative to redirect the beam of light and thereby cause the focus point to scan in an x direction;
a z-axis scanner operative to move the objective in a z direction parallel to an optical axis of the objective,
a first light detector arranged to receive light returning through the objective from the focus point and to generate an output signal representing an intensity of the light returning,
a tissue stabilizing mechanism coupled to the objective by way of a movable stage operative to move tissue relative to the objective in a y direction that is transverse relative to the x direction,
a controller configured to generate coordinated driving signals for the scanner unit and the z-axis scanner to cause the focus point to be raster scanned in an xz plane and control signals for the movable stage to cause the focus point to be scanned in the y direction,
a frame grabber operative to monitor the output signal in coordination with the raster scanning of the focus point to build a first series of xz image frames, and
a data processor connected to receive the xz image frames and configured to process the xz image frames to provide volumetric image data for a volume scanned by the focus point; and
a z-axis position sensor operative to output a signal indicative of a displacement of the objective in the z direction wherein the processing the set of xz plane image frames comprises performing distortion compensation, the distortion compensation comprising monitoring the output of the z-axis position sensor to determine an actual trajectory of the objective during the scanning along the z axis and shifting pixels of the xz image frames according to the actual trajectory.
19. The apparatus according to claim 18 wherein the scanning along the z axis is bidirectional, the xz plane image frames include first xz plane image frames acquired while scanning the objective in a first direction along the z axis and second xz plane image frames acquired while scanning the objective in a second direction opposed to the first direction along the z axis.
20. The apparatus according to claim 18 wherein the data processor is configured to monitor the output of the z-axis position sensor to determine a first actual trajectory of the objective during the scanning along the z axis in the first direction and a second actual trajectory of the objective during the scanning along the z axis in the second direction and to shift the positions of the pixels of the first xz plane image frames in the z direction based on the first actual trajectory and to shift the pixels of the second xz plane images in the z direction based on the second actual trajectory.
21. Apparatus for imaging tissue, the apparatus comprising:
a laser light source operative to emit a beam of light;
an objective arranged to receive and focus the beam of light to a focus point;
a scanner unit in an optical path between the laser light source and the objective, the scanner unit operative to redirect the beam of light and thereby cause the focus point to scan in an x direction;
a z-axis scanner operative to move the objective in a z direction parallel to an optical axis of the objective,
a first light detector arranged to receive light returning through the objective from the focus point and to generate an output signal representing an intensity of the light returning,
a tissue stabilizing mechanism coupled to the objective by way of a movable stage operative to move tissue relative to the objective in a y direction that is transverse relative to the x direction,
a controller configured to generate coordinated driving signals for the scanner unit and the z-axis scanner to cause the focus point to be raster scanned in an xz plane and control signals for the movable stage to cause the focus point to be scanned in the y direction,
a frame grabber operative to monitor the output signal in coordination with the raster scanning of the focus point to build a first series of xz image frames, and
a data processor connected to receive the xz image frames and configured to process the xz image frames to provide volumetric image data for a volume scanned by the focus point;
wherein the data processor is configured to perform motion compensation, the motion compensation comprising processing the xz plane image frames to determine representative z-axis locations of a tissue surface in each of the xz plane images and to shift the pixels of the xz plane images based on the representative z-axis locations.
22. The apparatus according to claim 21 wherein the data processor is configured to determine differences between the representative z-axis locations for adjacent ones of a sequence of the xz plane image frames, for each of the xz plane image frames in the sequence calculating a cumulative difference of the representative z-axis locations relative to a selected one of the xz plane images and shifting the pixels of each of the xz plane image frames in the sequence by the corresponding cumulative difference.