Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
An imaging system, such as a surgical microscope, laparoscope, or endoscope or integrated with these devices, includes an illuminator providing patterned white light and/or fluorescent stimulus light. The system receives and images light hyperspectrally, in embodiments using a hyperspectral imaging array, and/or using narrowband tunable filters for passing filtered received light to an imager. Embodiments may construct a 3-D surface model from stereo images, and will estimate optical properties of the target using images taken in patterned light or using other approximations obtained from white light exposures. Hyperspectral images taken under stimulus light are displayed as fluorescent images, and corrected for optical properties of tissue to provide quantitative maps of fluorophore concentration. Spectral information from hyperspectral images is processed to provide depth of fluorophore below the tissue surface. Quantitative images of fluorescence at depth are also prepared. The images are displayed to a surgeon for use in surgery.
1. A method of observing fluorescence and estimating depth of fluorophores operable on an optical and image processing system comprising:
providing, in a first mode, light from a light source at a fluorescent stimulus wavelength;
providing, in a second mode, broad-spectrum light from the light source;
spatially modulating light from the light source operating in the second mode to form spatially modulated light;
projecting onto tissue the spatially modulated light;
forming fluorescent stimulus wavelength images of the tissue as illuminated with the spatially modulated light, the spatially modulated light modulated in at least a first and a second predetermined spatial-light modulation pattern, where each spatial light pattern provides light varying in intensity across the tissue according to a spatial frequency;
extracting optical parameters at the fluorescent stimulus wavelength of the tissue from the fluorescent stimulus wavelength images of the tissue;
forming first fluorescent emissions wavelength images of the tissue at a first emissions wavelength as illuminated with the spatially modulated light;
extracting optical parameters at the first fluorescent emissions wavelength of the tissue from the fluorescent emissions wavelength images of the tissue;
forming second emissions wavelength images of the tissue at the first emissions wavelength with the light source operating in the first mode;
forming third emissions wavelength images of the tissue at a second emissions wavelength with the light source operating in the first mode;
determining a relationship between depth and ratios of intensity in the second emissions wavelength images and the third emissions wavelength images for a fluorophore in tissue, the first emissions wavelength and the second emissions wavelength being different;
determining a depth of the fluorophore at each pixel of a fluorescent emissions wavelength image selected from the second emissions wavelength images and the third emissions wavelength images based upon the relationship between the depth and the ratios of intensity in the second emissions wavelength images and the third emissions wavelength images; and
quantifying the fluorophore at each pixel of the selected fluorescent emissions wavelength image by using the optical parameters at the fluorescent stimulus wavelength and the optical parameters at the first emissions wavelength to correct intensity of the second emissions wavelength images to produce maps of quantitative fluorophore concentrations.
2. The method of claim 1 further comprising:
coregistrating a three-dimensional model derived from preoperative MRI or CT imaging to three-dimensional surface models extracted from stereo optical cameras; and
coregistrating the maps of quantitative fluorophore concentrations to the three-dimensional model derived from preoperative MRI or CT imaging.
3. The method of claim 2 further comprising locating tumors in the preoperative MRI or CT imaging to provide tumor locations; and
using the tumor locations identified in the preoperative MRI or CT imaging in estimating the maps of quantitative fluorophore concentration in the tumors and surrounding tissue.
4. The method of claim 2 where the maps of quantitative fluorophore concentration are used to improve estimates of tumor localization derived from the coregistered three-dimensional model derived from preoperative MRI or CT imaging and the three dimensional surface models extracted from the stereo optical cameras.