Multi-modal microscopic imaging
An adapter configured to be optically coupled to a plurality of microscopes and having (i) a first microscope interface configured to optically couple a first microscope system to an optical element that is in optical communication with an optical probe to provide first imaging data of a sample, and (ii) a second microscope interface configured to optically couple a second microscope system to the optical element to provide second imaging data of the sample. An optical imaging apparatus and method utilizing such adapter.
1 . An optical imaging apparatus configured to carry out multi-modal imaging of an internal tissue within a subject, the apparatus comprising:
an adapter comprising (i) a first microscope interface configured to optically couple a first microscope system to an optical element inside the adapter to provide first imaging data of the internal tissue, and (ii) a second microscope interface configured to optically couple a second microscope system to the optical element to provide second imaging data of the internal tissue;
a processor configured to co-register a first overlaid image generated by processing the first imaging data received through the adapter with a second overlaid image generated by processing the second imaging data received through the adapter with the use of aligning at least a portion of the first overlaid image and the second overlaid image,
wherein said aligning is carried out based at least in part on a coordinate transformation,
wherein the coordinate transformation is derived based at least in part on a difference in sizes, shapes, positions, or orientations of a landmark feature present in the first overlaid image and the second overlaid image;
and
an optical assembly that is coupled with the adapter at an interface of the first and second microscope interfaces and operably cooperated with the processor,
wherein the optical assembly includes:
(i) a portion of an illumination optical arm of a chosen microscope system of the first and second microscope systems, said portion of the illumination optical arm comprising one or more illumination light-emitting elements and configured to direct illumination light to said internal tissue within a specified field-of-view (FOV); and
(ii) a portion of a stimulation optical arm of the chosen microscope system comprising one or more stimulation light-emitting elements and configured to direct stimulation light to at least one of a first portion of said internal tissue within the specified FOV, a second portion of said internal tissue outside of the specified FOV, and to a third portion of said internal tissue that is partially within the specified FOV.
2 . An apparatus according to claim 1 , wherein the processor is operably cooperated with a tangible non-transitory storage medium that comprises software-encoded instructions which, when loaded onto the processor, cause the processor
(a) to govern an operation of said one or more illumination light-emitting elements to modulate the illumination light in a time-dependent and/or spatially-dependent fashion; and/or
(b) to govern an operation of said one or more stimulation light-emitting elements to modulate the stimulation light in a time-dependent and/or spatially-dependent manner; and/or
(c) to adjust an effective focal length of the chosen microscope system.
3 . An apparatus according to claim 2 , wherein said software-encoded instructions are configured to cause the processor:
to deliver said stimulation light to said internal tissue in a spatial pattern; and/or
to alter said spatial pattern based on a response of said tissue to said stimulation light; and/or
to modify the spatial pattern in response to detecting, within the specified FOV, an object of interest at said internal tissue based on analysis of an image of said internal tissue generated at an image sensor of the optical imaging apparatus.
4 . An apparatus according to claim 1 , wherein the processor is configured to generate the second overlaid image by:
using structural data representing the internal tissue to generate a multi-photon structural image and/or confocal structural image,
using functional data to generate a multi-photon functional image and/or confocal functional image, and
respectively overlaying the multi-photon structural image and/or confocal structural image with the multi-photon functional image and/or confocal functional image.
5 . An apparatus according to claim 1 , wherein the processor is configured to use the difference in sizes, shapes, positions, or orientations of the landmark feature in the first overlaid image and in the second overlaid image to determine an amount of distortion or warping of the landmark feature between the first overlaid image and the second overlaid image.
6 . An apparatus according to claim 1 , wherein the processor is further configured to generate a third overlaid image from the first overlaid image and the second overlaid image after said co-registering.
7 . An apparatus according to claim 6 , wherein the processor is configured to set the first overlaid image to a first color channel and the second overlaid image to a second color channel different from the first color channel to generate the third overlaid image.
8 . An apparatus according to claim 1 , wherein the processor is configured to generate the first overlaid image by obtaining a first cell map, constructing a structural image from a projection of a motion corrected movie, and overlaying the first cell map with the structural image.
9 . An apparatus according to claim 8 , wherein the constructing a structural image includes constructing the structural image from at least one of a maximum projection of the motion-corrected movie and a variance projection of the motion-corrected movie.
10 . An apparatus according to claim 1 , wherein the processor is configured to train a neural network to determine how a size, a shape, a position, or an orientation of the landmark feature is altered between different optical imaging modalities present in the apparatus.
11 . An apparatus according to claim 1 , further comprising an optical probe that is optically and/or mechanically coupled with the optical element and the optical assembly through an interface of the first and second interfaces and that is configured to deliver the illumination light and the stimulation light to said internal tissue without transmitting said illumination light and/or said stimulation light through a biological barrier surrounding said subject.
12 . An apparatus according to claim 11 , wherein the optical probe is configured to be in direct contact with the internal tissue while delivering the illumination light and the stimulation light to said internal tissue without transmitting said illumination light and/or said stimulation light through the biological barrier surrounding the subject.
13 . An apparatus according to claim 11 , wherein said optical probe possesses non-zero optical power.
14 . An apparatus according to claim 11 ,
wherein the optical probe is configured to collect light reflected, scattered, or emitted by said internal tissue and relay said light to an imaging optical arm of the apparatus, and/or
wherein the optical probe is configured as a lens or lens system at least partially implantable in said subject, and/or
wherein said optical probe is configured as an optical component shared by said illumination optical arm and said stimulation optical arm.
15 . An apparatus according to claim 1 , wherein the optical assembly further includes a portion of an imaging optical arm of the chosen microscope system, said portion of the imaging optical arm comprising an image sensor and configured to receive imaging light including light reflected, scattered, or emitted by said tissue to generate an image of the tissue within the specified FOV.
16 . An apparatus according to claim 15 , further comprising an optical probe that is:
(a) optically and/or mechanically coupled with the optical assembly through an interface of the first and second interfaces and configured to deliver the illumination light and the stimulation light to said internal tissue without transmitting said illumination light and/or said stimulation light through a biological barrier surrounding said subject, and
(b) configured as an optical component shared by a combination of any of said illumination optical arm, said stimulation optical arm, and said imaging optical arm.
17 . An apparatus according to claim 15 , wherein the illumination optical arm, the stimulation optical arm, and the imaging optical arm share at least one deformable lens.
18 . An apparatus according to claim 1 , wherein at least one of the following conditions is satisfied:
(a) the first microscope system includes a one-photon microscope or a multi-photon microscope and/or the second microscope system includes a confocal microscope; and
(b) the first microscope system and the second microscope system include microscopes of different types.
19 . An apparatus according to claim 1 , wherein the first microscope interface is configured to permit the adapter to bear weight of the first microscope system and/or wherein the second microscope interface is configured to permit the adapter to bear weight of the second microscope system.
20 . An apparatus according to claim 1 , further comprising a corrective optical element optically cooperated with an element of the optical assembly to achieve a spatial resolution of better than 2 μm at a center of the specified FOV.
21 . A method for multi-modal imaging of an internal tissue of a subject, the method comprising:
removably attaching a housing of the optical assembly of the optical imaging apparatus according to claim 1 to the adapter, which adapter has been connected to the subject in a fixed position, to align the chose microscope system with respect to the internal tissue;
receiving the first imaging data and the second imaging data representing the internal tissue through the adapter at least in part by delivering at least one of (i) the illumination light from the one or more illumination light-emitting elements through the portion of the illumination optical arm of said chosen microscope system, and (ii) the stimulation light from the one or more stimulation light-emitting elements through the portion of the stimulation optical arm of said chosen microscope system to the internal tissue; and
with the use of said processor:
processing the first imaging data, received through the adapter, to generate a first overlaid image,
processing the second imaging data, received through the adapter, to generate a second overlaid image, and
co-registering the first overlaid image and the second overlaid image with the use of aligning at least a portion of the first overlaid image and the second overlaid image based at least in part on a coordinate transformation, wherein the coordinate transformation is derived based at least in part on a difference in sizes, shapes, positions, or orientations of a landmark feature present in the first overlaid image and the second overlaid image.
22 . A method according to claim 21 , wherein said receiving the first imaging data and the second imaging data includes acquiring, at an image sensor of the chosen microscope system of the first and second microscope systems, light that has been reflected, scattered, or emitted by said internal tissue to generate an image of said internal tissue.
23 . A method according to claim 22 , wherein said acquiring includes acquiring the light through a corrective optical element optically cooperated with an element of the optical assembly to achieve a spatial resolution of higher than 2 μm at a center of the specified FOV.
24 . A method according to claim 21 , wherein said delivering includes transmitting light through an optical probe that is optically and/or mechanically coupled with the optical assembly through an interface of the first and second microscope interfaces.
25 . A method according to claim 21 , wherein:
said processing the first imaging data includes processing functional imaging data, and
said processing the second imaging data includes processing structural imaging data;
and further comprising:
with the use of said processor, setting the first overlaid image to a first color channel of the apparatus and the second overlaid image to a second color channel of the apparatus, the first and second color channels being different from one another, and generating a third overlaid image from the first overlaid image and the second overlaid image after said co-registering.
26 . A method according to claim 21 , wherein said delivering is carried out without transmitting the illumination light and/or the stimulation light through a biological barrier surrounding said subject.
27 . A method according to claim 21 , wherein said delivering includes transmitting light through an optical probe of said microscope system, wherein the optical probe includes a GRIN lens.
28 . A method according to claim 21 , wherein the optical assembly comprises an imaging optical arm, and wherein:
(a) said receiving first imaging data and second imaging data includes receiving the first and second imaging data through a portion of the imaging optical arm, said portion of the imaging optical arm comprising an image sensor and configured to receive light reflected, scattered, or emitted by said tissue to generate an image of the tissue within the specified FOV;
and/or
(b) said receiving first imaging data and second imaging data includes transmitting light through one or more deformable lenses configured to adjust a corresponding focal depth of at least one of the illumination optical arm, the stimulation optical arm, and the imaging optical arm.
29 . A method according to claim 21 , further comprising:
modulating the illumination light in a time-dependent and/or spatially-dependent fashion; and/or
modulating the stimulation light in a time-dependent and/or spatially-dependent manner; and/or
adjusting an effective focal length of the chosen microscope system of the first and second microscope systems.