Apparatuses and methods involving multi-modal imaging of a sample

1 . An apparatus comprising:

a light source configured to output a light beam along an optical pathway;

a first polarizer coupled to the light source and configured to pass first polarized light from the output light beam and toward a sample along the optical pathway;

an image sensor, including circuitry, configured to collect light reflected from the sample in response to the passed first polarized light;

a second polarizer arranged along the optical pathway between the sample and the image sensor and configured to pass second polarized light from the light reflected from the sample in response to the passed first polarized light and toward the image sensor, wherein the first and second polarizers are linear and cross each other;

a filter arranged along the optical pathway and configured to selectively pass the reflected light in a visible light range and near infrared range (NIR) light range toward the image sensor;

control circuitry configured and arranged with the image sensor to image the sample by:

causing the first polarizer and the second polarizer to adjust to different polarization angles with respect to one another; and

collecting image data of the sample from the light reflected while the first and second polarizers are at the different polarization angles with respect to one another and while the filter selectively passes the visible light and the NIR light ranges of the second polarized light; and

processing circuitry configured to:

generate a plurality of NIR image frames and a plurality of visible light image frames from the image data collected while the first and second polarizers are at the different polarization angles;

revise the image data including:

adjusting contrast of the plurality of NIR image frames and the plurality of visible light image frames by redistributing light intensity values; and

aligning the plurality of NIR image frames with the plurality of visible light image frames by identifying keypoints in the respective image frames and shape descriptors of features, sorting the keypoints based on the shape descriptors, and aligning pixels within the respective image frames using the sorted keypoints; and

fuse the plurality of NIR image frames and the plurality of visible light image frames into a single composite image based on the revised image data.

2 . The apparatus of claim 1 , wherein:

the first polarized light and the second polarized light are associated with a slant with respect to one another, the sample includes a tissue sample, and

the control circuitry is configured to cause the first polarizer and the second polarizer to adjust to the different polarization angles such that the first polarized light and second polarized light are crossed with each other and for the different polarization angles, resulting in optical reflections of birefringence from portions of the tissue sample to be focused or discriminated when aligned to a polarization of collimated incident light.

3 . The apparatus of claim 1 , wherein the control circuitry is configured to collect the image data by causing the filter to selectively pass the visible light and the NIR light and collecting a plurality of image frames of the sample while the first and second polarizers are at the different polarization angles with respect to one another, while the filter selectively passes the visible light range and while the filter selectively passes the NIR light range, wherein the plurality of NIR image frames are generated from the NIR light range of 715 nanometers (nm) to 2500 nm and the plurality of visible light image frames are generated from the visible light range of 400 nm to 700 nm.

4 . The apparatus of claim 1 , wherein the control circuitry is configured to collect the image data by collecting a sequential order of image frames including the plurality of NIR image frames and the plurality of visible light image frames responsive to the first and second polarizers being at the different polarization angles with respect to one another, and while the filter selectively passes both the NIR and visible light ranges.

5 . The apparatus of claim 1 , wherein the processing circuitry is configured to adjust the contrast of the plurality of NIR image frames and the plurality of visible light image frames by redistributing the light intensity values, applying a bilateral filter and combining the redistributed light intensity values with the bilateral filtered space.

6 . The apparatus of claim 1 , wherein the processing circuitry is further configured to identify which of the different polarization angles of the first and second polarizers results in areas of interest of the sample being in focus, and to provide feedback to the control circuitry to revise the image data based on the areas of interest being in focus.

7 . The apparatus of claim 1 , wherein the filter includes a first bandpass filter configured to selectively pass visible light and a second bandpass filter configured to selectively pass NIR wavelengths.

8 . The apparatus of claim 7 , further including a motorized rotator arranged with the first and second bandpass filters, and the control circuitry is further configured and arranged to selectively rotate the motorized rotator such that one of the first and second bandpass filters are arranged in the optical pathway to selectively pass one of the visible and NIR wavelengths.

9 . The apparatus of claim 1 , wherein the filter includes a bandpass filter configured to selectively block incident light and a color filter array configured to capture NIR, red, green, and blue channels at the same time.

10 . The apparatus of claim 9 , wherein the control circuitry is configured to collect the image data by capturing first image data using collimated incident light as generated by the first and second polarizer, and by capturing second image data using non-polarized light from the light source, the captured first and second image data including the plurality of NIR image frames and the plurality of visible light image frames of the sample which are captured at the same time.

11 . The apparatus of claim 1 , further including a first motorized rotator coupled to the first polarizer and a second motorized rotator coupled to the second polarizer, wherein the control circuitry is further configured to selectively rotate the first and second motorized rotators such that the first and second polarizers are at the different polarization angles.

12 . The apparatus of claim 11 , wherein the first and second polarizers each include a plurality of polarized filters arranged on the first and second motorized rotators to provide the different polarization angles and each include an all-pass filter arranged on the first and the second motorized rotators, the all-pass filters being configured to pass light of all polarizations.

13 . The apparatus of claim 1 , wherein the apparatus includes an endoscope having a distal end, a proximal end, and a rigid tube arranged between the distal end and the proximal end, wherein:

the image sensor is coupled to the distal end, and the distal end is arranged with the rigid tube to provide light reflected from the sample to the image sensor;

the light source is coupled to the rigid tube via another tube between the distal end and the proximal end and to provide the light beam along the optical pathway;

the first polarizer is positioned proximal to the light source, the first polarizer including a filter that is sized according to illumination channels associated with the other tube and the light source;

the second polarizer is positioned proximal to the image sensor along the rigid tube, the second polarizer including a filter that is sized according to an imaging channel associated with the rigid tube and the image sensor; and

the proximal end is configured to couple to an elongated flexible tube.

14 . The apparatus of claim 1 , wherein the processing circuitry is in communication with the control circuitry and configured to:

revise the image data to improve the contrast of areas of interest in the image data of the sample, and thereby increasing salient features of the areas of interest;

identify locations of the areas of interest in the revised image data by the aligning of the image data including the plurality of NIR image frames with the plurality of visible light image frames and identifying objects and object segments in the image data; and

fuse the plurality of NIR image frames and the plurality of visible light image frames into the single composite image of the sample based on the revised image data and the identified locations of the areas of interest.

15 . A method comprising:

adjusting a first polarizer and a second polarizer to different polarization angles with respect to one another, wherein the first and second polarizers are linear and cross each other while at each of the different polarization angles:

outputting a light beam along an optical pathway using a light source;

passing, using the first polarizer, first polarized light from the output light beam and toward a sample along the optical pathway;

reflecting light from the sample responsive to the first polarized light;

passing second polarized light, using the second polarizer, from the light reflected from the sample and toward an image sensor;

selectively passing the second polarized light in a visible light range and a near infrared range (NIR) light range toward the image sensor;

collecting, using the image sensor, image data of the sample from the light reflected while the first and second polarizers are at the different polarization angles with respect to one another and while the visible light and the NIR light ranges of the second polarized light are selectively passed;

generating a plurality of NIR image frames and a plurality of visible light image frames from the image data collected while the first and second polarizers are at the different polarization angles;

revising the image data including:

adjusting contrast of the plurality of NIR image frames and the plurality of visible light image frames by redistributing light intensity values; and

aligning the plurality of NIR image frames with the plurality of visible light image frames by identifying keypoints in the image frames and shape descriptors of features, sorting the keypoints based on the shape descriptors, and aligning pixels within respective image frames using the sorted keypoints; and

fusing the plurality of NIR image frames and the plurality of visible light image frames into a single composite image using the revised image data.

16 . The method of claim 15 , wherein fusing the plurality of NIR image frames and the plurality visible light image frames into the single composite image includes:

combining the plurality of NIR image frames to form a composite NIR image;

combining the plurality of visible light image frames to form a composite visible light image; and

combining the composite NIR image and the composite visible light image to form the single composite image.

17 . The method of claim 15 , wherein adjusting the first polarizer and the second polarizer to the different polarization angles results in optical reflections of birefringence from portions of the sample to be focused or discriminated when aligned to polarization of collimated incident light, and the method further including identifying which of the different polarization angles of the first and second polarizers results in an area of interest of the sample being in focus.

18 . The method of claim 15 , wherein:

adjusting the contrast of the plurality of visible light image frames and the plurality of NIR image frames to revise the image data further includes computing histograms, redistributing the light intensity values based on the histograms, applying a bilateral filter, and combining the redistributed light intensity values with the bilateral filtered space; and

aligning the plurality of NIR image frames with the plurality of visible light image frames further includes identifying the keypoints in the image frames and the shape descriptors of features, using the shape descriptors to sort the keypoints, and estimating a homography using the sorted keypoints and to align pixels within respective image frames.

19 . The method of claim 15 , wherein collecting the image data includes:

causing a filter to selectively pass the visible light range and the NIR light range; and

collecting a plurality of image frames of the sample while the first and second polarizers are at the different polarization angles with respect to one another, while the filter selectively passes the visible light range and while the filter selectively passes the NIR light range.

20 . The method of claim 15 , wherein:

collecting the image data includes collecting a sequential order of image frames responsive to the first and second polarizers being at the different polarization angles with respect to one another, and while a filter selectively passes both the NIR and visible light ranges; and

adjusting the contrast of the plurality of NIR image frames and the plurality of visible light image frames includes redistributing the light intensity values, applying a bilateral filter, and combining the redistributed light intensity values with the bilateral filtered space.

21 . The method of claim 15 , wherein collecting the image data includes capturing first image data using collimated incident light as generated by the first and second polarizer and capturing second image data using non-polarized light from the light source, the captured first and second image data including video of the sample.