Method of multiple image reconstruction and registration
Systems and methods related to combing multiple images are disclosed. An example method of combining multiple images of a body structure includes capturing a first input image with a digital camera positioned at a first location at a first time point, representing the first image with a first plurality of pixels, capturing a second input image with the digital camera positioned at a second location at a second time point, representing the second image with a second plurality of pixels, generating a first feature distance map of the first input image, generating a second feature distance map of the second input image, calculating the positional change of the digital camera between the first time point and the second time point and utilizing the first feature distance map, the second feature distance map and the positional change of the digital camera to generate a three-dimensional surface approximation the body structure.
1 . A method of combining multiple images of a body structure, the method comprising:
capturing a first input image with a digital camera disposed along an endoscope inserted in a body lumen of a patient, wherein the digital camera is positioned at a first location within the body lumen at a first time point;
representing the first image with a first plurality of pixels;
capturing a second input image with the digital camera positioned at a second location within the body lumen at a second time point;
representing the second image with a second plurality of pixels;
generating a first feature distance map of the first input image;
generating a second feature distance map of the second input image;
calculating the positional change of the digital camera between the first time point and the second time point; and
utilizing the first feature distance map, the second feature distance map and the positional change of the digital camera to generate a three-dimensional surface approximation of the body structure within the body lumen;
wherein generating the first feature distance map includes assigning a numerical value to the one or more pixels of the first plurality of pixels;
wherein generating the second feature distance map includes assigning a numerical value to the one or more pixels of the second plurality of pixels; and
wherein utilizing the first feature distance map, the second feature distance map and the positional change of the digital camera to generate a three-dimensional surface approximation of the body structure includes calculating a transformation between the first input image and the second input image using a chamfer matching registration process.
2 . The method of claim 1 , wherein the first image corresponds to the body structure, and wherein generating the first feature distance map includes selecting one or more pixels from the first plurality of pixels, wherein the one or more pixels from the first plurality of pixels are selected based on their proximity to a feature of the first image.
3 . The method of claim 2 , wherein the one or more pixels from the first plurality of pixels are selected based on their proximity to a central longitudinal axis of the body structure.
4 . The method of claim 3 , wherein the second image corresponds to the body structure, and wherein generating the second feature distance map includes selecting one or more pixels from the second plurality of pixels, wherein the one or more pixels from the second plurality of pixels are selected based on their proximity to a feature of the second image.
5 . The method of claim 4 , wherein the one or more pixels from the second plurality of pixels are selected based on their proximity to a central longitudinal axis of the body structure.
6 . The method of claim 1 , wherein generating the first feature distance map includes calculating rectilinear distances from a portion of the body structure to one or more pixels of the first image and generating the second feature distance map includes calculating rectilinear distances from a portion of the body structure to the one or more pixels of the second image.
7 . The method of claim 1 , wherein the first plurality of pixels are arranged in a first coordinate grid, and wherein second plurality of pixels are arranged in a second coordinate grid, and wherein the coordinate locations of the first plurality of pixels are at the same respective locations as the coordinate locations of the second plurality of pixels.
8 . The method of claim 1 , further comprising generating a hybrid feature distance map by registering the first feature distance map with the second feature distance map using one or more degrees of freedom corresponding to a digital camera motion configuration parameter.
9 . The method of claim 8 , wherein the digital camera motion parameter includes one or more of a positional change and a rotational change of the digital camera along a scope axis.
10 . The method of claim 8 , further comprising assessing the confidence of the hybrid distance map by comparing the value of distances calculated in the hybrid distance map to a threshold distance value.
11 . The method of claim 1 , wherein the first plurality of pixels includes a feature pixel, and wherein the feature pixel corresponds to a feature of the first image.
12 . The method of claim 11 , wherein the first plurality of pixels are each represented by a color, and wherein the color of the feature pixel is darker than the color of each of the other pixels of the first plurality of pixels.
13 . The method of claim 1 , wherein the chamfer matching registration process includes registering the first input image with the second input image using four degrees of freedom of the endoscope, and wherein three or more of the four degrees of freedom is selected from the group comprising the advancement of the endoscope, the withdrawal of the endoscope, the rotation of the endoscope, the flex of the endoscope and the flex angle of the endoscope.
14 . A method of combining multiple images of a body structure, the method comprising:
using an image capture device of an endoscope to obtain a first image within a body lumen of a patient at a first time point and to obtain a second image within the body lumen of the patient at a second time point, wherein the image capture device is positioned at a first position when it captures the first image at the first time point, and wherein the image capture device is positioned at a second position when it captures the second image at the second time point, and wherein the second time point occurs after the first time point;
representing the first image with a first plurality of pixels;
representing the second image with a second plurality of pixels;
generating a first feature distance map of the first input image;
generating a second feature distance map of the second input image;
calculating the positional change of image capture device between the first time point and the second time point; and
utilizing the first feature distance map, the second feature distance map and the positional change of image capture device to generate a three-dimensional surface approximation of the body structure within the body lumen of the patient;
wherein generating the first feature distance map includes assigning a numerical value to the one or more pixels of the first plurality of pixels;
wherein generating the second feature distance map includes assigning a numerical value to the one or more pixels of the second plurality of pixels; and
wherein utilizing the first feature distance map, the second feature distance map and the positional change of the image capturing device to generate a three-dimensional surface approximation of the body structure includes calculating a transformation between the first input image and the second input image using a chamfer matching registration process.
15 . The method of claim 14 , wherein the first image corresponds to the body structure, and wherein generating the first feature distance map includes selecting one or more pixels from the first plurality of pixels, wherein the one or more pixels from the first plurality of pixels are selected based on their proximity to a features of the first image correlated to body structures.
16 . The method of claim 15 , wherein the one or more pixels from the first plurality of pixels are selected based on their proximity to a central longitudinal axis of the body structure.
17 . The method of claim 16 , wherein the second image corresponds to the body structure, and wherein generating the second feature distance map includes selecting one or more pixels from the second plurality of pixels, wherein the one or more pixels from the second plurality of pixels are selected based on their proximity to the central longitudinal axis of the body structure.
18 . The method of claim 14 , further comprising generating a hybrid feature distance map by registering the first feature distance map with the second feature distance map using a one or more degrees of freedom corresponding to a digital camera motion parameter and a scope state configuration parameter.
19 . The method of claim 14 , wherein the chamfer matching registration process includes registering the first input image with the second input image using four degrees of freedom of the endoscope, and wherein three or more of the four degrees of freedom is selected from the group comprising the advancement of the endoscope, the withdrawal of the endoscope, the rotation of the endoscope, the flex of the endoscope and the flex angle of the endoscope.
20 . A system for generating a fused image from multiple images, comprising:
a processor; and
a non-transitory computer-readable storage medium comprising code configured to perform a method of fusing images, the method comprising:
capturing a first input image containing a body structure with a digital camera positioned at a first location at a first time point;
representing the first image with a first plurality of pixels;
capturing a second input image containing the body structure with the digital camera positioned at a second location at a second time point;
representing the second image with a second plurality of pixels;
generating a first feature distance map of the first input image;
generating a second feature distance map of the second input image;
calculating the positional change of the digital camera between the first time point and the second time point; and
utilizing the first feature distance map, the second feature distance map and the positional change of the digital camera to generate a three-dimensional surface approximation of the body structure,
wherein generating the first feature distance map includes assigning a numerical value to the one or more pixels of the first plurality of pixels;
wherein generating the second feature distance map includes assigning a numerical value to the one or more pixels of the second plurality of pixels; and
wherein utilizing the first feature distance map, the second feature distance map and the positional change of the digital camera to generate a three-dimensional surface approximation of the body structure includes calculating a transformation between the first input image and the second input image using a chamfer matching registration process.