Endoscope system, lumen structure calculation system, and method for creating lumen structure information
An endoscope system includes an insertion section, an imager, and a control device including a processor that processes a signal from the imager. The insertion section is inserted into a lumen as an object. The imager is monocular and is provided in the insertion section to capture an image of the object. The processor acquires actual size determination information for determining an actual size of at least a portion of the lumen. The processor calculates a three-dimensional structure of the lumen based on the captured image. The processor also calculates three-dimensional structure size information for determining an actual size of at least a portion of the three-dimensional structure by calibrating the three-dimensional structure of the lumen based on the actual size determination information. The processor also outputs specific portion size information representing an actual size of a specific portion of the three-dimensional structure, based on the three-dimensional structure size information.
1 . An endoscope system comprising:
an insertion section configured to be inserted into a lumen as an object;
a monocular imager provided in the insertion section and configured to capture an image of the object; and
a control device including a processor that processes a signal from the imager, wherein the processor is configured to:
acquire actual size determination information for determining an actual size of at least a portion of the lumen;
determine a three-dimensional structure of the lumen based on the captured image;
determine three-dimensional structure size information for determining an actual size of at least a portion of the three-dimensional structure by calibrating the three-dimensional structure of the lumen based on the actual size determination information;
determine a line-of-sight direction with respect to a specific portion, the specific portion being a portion of the lumen for measurement; and
output specific portion size information representing an actual size of the specific portion of the three-dimensional structure and a height from an inner wall of the lumen to a maximum extent of the specific portion along a normal line extending from the inner wall of the lumen, based on the three-dimensional structure size information.
2 . The endoscope system as defined in claim 1 , wherein
the processor acquires a size on the captured image of an object for comparison, which appears in the captured image and has a known size, as the actual size determination information.
3 . The endoscope system as defined in claim 1 , wherein
the processor
extracts a plurality of feature points from each captured image,
calculates a position of each feature point in a three-dimensional space, based on positions of the plurality of feature points on the captured image, and
calculates the three-dimensional structure of the lumen, based on the calculated position of each feature point in the three-dimensional space.
4 . The endoscope system as defined in claim 3 , wherein
the processor
extracts the specific portion from the captured image and extracts the feature points from the captured image including the specific portion, and
associates the extracted feature points with the feature points in the three-dimensional structure to set a position of the specific portion in the three-dimensional structure.
5 . The endoscope system as defined in claim 3 , wherein
the imager acquires the captured image at a plurality of time points, and
the processor
extracts the feature points that commonly appear in the captured images at the plurality of time points,
acquires three-dimensional arrangement information of the insertion section at the plurality of time points based on output of a position sensor that extracts information of at least a portion of position and orientation of the imager, and
calculates positions of the feature points in the three-dimensional space based on the three-dimensional arrangement information of the insertion section at the plurality of time points and positions on the captured image of the feature points that commonly appear in the captured images at the plurality of time points.
6 . The endoscope system as defined in claim 1 , wherein
the processor sets the specific portion based on the captured image.
7 . The endoscope system as defined in claim 6 , wherein
the processor presents the specific portion on the captured image in a manner visible to a user.
8 . The endoscope system as defined in claim 6 , wherein
the processor automatically sets the specific portion.
9 . The endoscope system as defined in claim 6 , wherein
the captured image is separable into a plurality of items of color information, and
the processor
sets the specific portion using first color information included in the plurality of items of color information, and
calculates the three-dimensional structure and the three-dimensional structure size information using second color information that is included in the plurality of items of color information and does not include the first color information.
10 . The endoscope system as defined in claim 6 , wherein
a plurality of setting modes are available to select, and
the processor sets the specific portion based on the selected setting mode.
11 . The endoscope system as defined in claim 6 , wherein
the processor sets the specific portion based on setting that has been redone in a different setting mode of the same image or using a different image.
12 . The endoscope system as defined in claim 1 , wherein
the processor determines the three-dimensional structure and the three-dimensional structure size information based on the captured image captured within a predetermined time and the actual size determination information acquired within the predetermined time.
13 . The endoscope system as defined in claim 1 , wherein
the processor outputs the actual size of a target object projected on a plane parallel to the lumen around the specific portion as the specific portion size information.
14 . The endoscope system as defined in claim 1 , wherein
the processor outputs the actual size of a target object in a direction perpendicular to the lumen around the specific portion as the specific portion size information.
15 . The endoscope system as defined in claim 1 , wherein
the processor determines three-dimensional structure information with the actual size based on the actual size determination information.
16 . The endoscope system as defined in claim 1 , wherein
the processor outputs the specific portion size information in association with information of the three-dimensional structure of the specific portion.
17 . The endoscope system as defined in claim 1 , wherein the major diameter, the minor diameter, and the height of the specific portion are output based on an object projected on a plane perpendicular to the normal line from a plane parallel to the inner wall of the lumen around the specific portion.
18 . The endoscope system as defined in claim 17 , wherein the height is based on a plane parallel to the inner wall, and is the length of the specific portion along a direction parallel to the normal vector.
19 . The endoscope system as defined in claim 1 , wherein, as the actual size, a distance from a predetermined location, representing an anatomical entry point of the lumen, to the specific portion is calculated and output.
20 . The endoscope system as defined in claim 1 , wherein the specific portion is displayed on a monitor to the user while changing the line-of-sight direction.
21 . The endoscope system as defined in claim 1 , wherein the actual size of the three-dimensional structure is calculated.
22 . The endoscope system as defined in claim 1 , wherein the actual size of the three-dimensional structure and the actual size of the specific portion are displayed on a monitor at the same time.
23 . A lumen structure calculation system comprising a processor including hardware, wherein the processor;
acquires a captured image of an object acquired by a monocular imager provided in an insertion section, which is inserted into a lumen as the object, and actual size determination information, which is information for determining an actual size of at least a portion of the lumen;
determines a three-dimensional structure of the lumen based on the captured image;
determines three-dimensional structure size information for determining an actual size of at least a portion of the three-dimensional structure by calibrating the three-dimensional structure of the lumen based on the actual size determination information;
determine a line-of-sight direction with respect to a specific portion, the specific portion being a portion of the lumen for measurement; and
outputs specific portion size information representing an actual size of the specific portion of the three-dimensional structure and a height from the inner wall of the lumen to a maximum extent of the specific portion along a normal line extending from the inner wall of the lumen, based on the three-dimensional structure size information.
24 . A method for creating lumen structure information comprising:
acquiring a captured image of an object acquired by a monocular imager provided in an insertion section, which is inserted into a lumen as the object;
acquiring actual size determination information, which is information for determining an actual size of at least a portion of the lumen;
determining a three-dimensional structure of the lumen based on the captured image;
determining three-dimensional structure size information for determining an actual size of at least a portion of the three-dimensional structure by calibrating the three-dimensional structure of the lumen based on the actual size determination information;
determining a line-of-sight direction with respect to a specific portion, the specific portion being a portion of the lumen for measurement; and
outputting specific portion size information representing an actual size of the specific portion of the three-dimensional structure and a height from the inner wall of the lumen to a maximum extent of the specific portion along a normal line extending from the inner wall of the lumen, based on the three-dimensional structure size information.