SYSTEM AND METHOD FOR MOSAICKING ENDOSCOPE IMAGES CAPTURED FROM WITHIN A CAVITY

1 . A method of imaging the interior surface of a cavity, comprising:

capturing images using an endoscope, where the optics of the endoscope are radially symmetrical;

locating the optical center of each of the captured images;

dewarping each of the captured images by mapping the image from polar coordinates centered on the optical center of the image to rectangular coordinates;

discarding portions of each dewarped image to create high clarity dewarped images;

estimating the motion of the endoscope with respect to the interior surface of the cavity that occurred between successive high clarity dewarped images;

registering the high clarity dewarped images with respect to each other using the estimates of motion; and

combining the registered high clarity dewarped images to create at least one mosaic.

2 . The method of claim 1 , wherein the optical center is located by comparing the captured image to a template image.

3 . The method of claim 2 , wherein the optical center is located by locating the translation between the captured image and the template image that results in the smallest sum of absolute differences.

4 . The method of claim 3 , wherein:

a large range of possible translations are considered in locating the optical center of a first captured image; and

a small range of possible translations are considered relative to the location of the optical center of the first captured image in locating the optical center of a second captured image.

5 . The method of claim 1 , wherein the endoscope includes a tip through which images are captured and the tip includes fiducial markings that assist in the location of the optical center of captured images.

6 . The method of claim 1 , wherein discarding portions of each dewarped image that possess insufficient image clarity to create high clarity dewarped images comprises discarding a predetermined portion of each dewarped image.

7 . The method of claim 1 , wherein discarding portions of each dewarped image that possess insufficient image clarity to create high clarity dewarped images comprises:

performing blur detection on each image; and

discarding at least one region of the image possessing blurriness exceeding a predetermined threshold.

8 . The method of claim 1 , further comprising adjusting the brightness of pixels to account for variations in the illumination of the imaged surface.

9 . The method of claim 1 , further comprising compensating for pixels within the captured images that are the result of known defects in the endoscope.

10 . The method of claim 1 , wherein estimating the motion of the endoscope with respect to the interior surface of the cavity that occurred between successive high clarity dewarped images comprises determining a motion vector at which the square of the differences between successive captured images is a minimum.

11 . The method of claim 10 , wherein determining the motion vector at which the square of the differences between successive captured images is a minimum comprises:

creating a Gaussian pyramid for each image; and

using the motion vector at which the square of the differences between images of corresponding lower resolution in the Gaussian pyramids is a minimum to determine the motion vector at which the square of the differences between images of corresponding higher resolution images in the Gaussian pyramids is a minimum, until the motion vector at which the square of the differences between the two captured images is a minimum is determined.

12 . The method of claim 1 , further comprising:

performing multiple passes over the captured images to improve the accuracy of the estimation of the motion of the endoscope with respect to the interior surface of the cavity compared to the initial estimate determined by comparing successive high clarity dewarped images; and

performing multiple passes to register successive images with respect to each other to reduce registration errors accumulated during initial sequential registration.

13 . The method of claim 1 , wherein images are captured at a frame rate chosen so that the motion that occurs between the captured images is sufficiently small for the high clarity dewarped images to overlap.

14 . The method of claim 1 , wherein:

the captured images are color images;

the system generates mosaics in real time; and

processing latency is reduced by converting the captured images from color to grayscale and locating the optical center of the grayscale images.

15 . The method of claim 1 , wherein processing latency is reduced by dewarping grayscale images and performing motion estimation using grayscale images.

16 . The method of claim 1 , further comprising performing image segmentation to identify segments of the image corresponding to different surfaces of the cavity and combining the image segments corresponding to different surfaces of the cavity to form separate mosaics of each of the different surfaces of the cavity.

17 . The method of claim 16 , wherein performing image segmentation further comprises locating at least the two darkest columns in the high clarity dewarped images.

18 . The method of claim 17 , further comprising limiting the rotation of the endoscope and assuming the two darkest columns are constrained to be located within defined regions of the high clarity dewarped images.

19 . The method of claim 18 , wherein the defined regions correspond to the two halves of the field of view of the endoscope.

20 . The method of claim 16 , comprising using boundaries between groups of aligned motion vectors of blocks of pixels within the image to identify segments of the image corresponding to different surfaces of the cavity.