IP Library › Granted Patent US 11,432,713
Granted Patent B2
US 11,432,713 · App. 16/672,736 · Granted Sep 6, 2022

Endoscope system, endoscope image processing method, and computer readable recording medium for generating a three-dimensional shape image

Inventor: Shunya Akimoto (Nishitokyo, JP)
Assignee: OLYMPUS CORPORATION
A61B1/05A61B1/00009G02B23/24G06T7/0012G06T2207/10068
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,432,713
App. No.
16/672,736
Granted
Sep 6, 2022
Kind
B2
Abstract

An endoscope system includes an endoscope configured to pick up an image of an inside of a subject to acquire an image, and a processor including hardware. The processor estimates a three-dimensional position of a target portion within the subject based on an image, sets for the three-dimensional position a reliability corresponding to a predetermined parameter related to the endoscope system determined when the image is acquired, and selects, when a plurality of three-dimensional positions for the target portion exist, a predetermined three-dimensional position among the plurality of three-dimensional positions depending on the reliability and generates a three-dimensional shape image.

Claims (43)

1. An endoscope system comprising:

an endoscope configured to pick up an image of an inside of a subject to acquire an image; and

a processor including hardware,

wherein the processor is configured to

estimate a three-dimensional position of a target portion within the subject based on a predetermined image acquired by the endoscope,

set, for the estimated three-dimensional position, a reliability corresponding to a predetermined parameter related to the endoscope system determined when the predetermined image is acquired, and

select, when a plurality of three-dimensional positions for the target portion exist, a predetermined three-dimensional position among the plurality of three-dimensional positions depending on the set reliability and generate a three-dimensional shape image based on the selected predetermined three-dimensional position.

2. The endoscope system according to claim 1 , wherein

the processor selects the three-dimensional position for which the reliability is highest as the predetermined three-dimensional position.

3. The endoscope system according to claim 2 , wherein

the processor sets one of the estimated plurality of three-dimensional positions as a three-dimensional position of interest, draws a straight line between a position of a distal end of the endoscope when the image based on which the three-dimensional position of interest is estimated is acquired and the three-dimensional position of interest, to compare a reliability for another three-dimensional position on the straight line with a reliability for the three-dimensional position of interest, and selects the three-dimensional position of interest or the other three-dimensional position for which the reliability is higher.

4. The endoscope system according to claim 3 , wherein

the processor sets the estimated plurality of three-dimensional positions to the three-dimensional position of interest in descending order from the three-dimensional position for which the reliability is highest.

5. The endoscope system according to claim 2 , wherein

the processor sets one of the estimated plurality of three-dimensional positions to a three-dimensional position of interest, sets a vicinity range of a size corresponding to a reliability for the three-dimensional position of interest, to compare a reliability for another three-dimensional position within the vicinity range with the reliability for the three-dimensional position of interest, and selects the three-dimensional position of interest or the other three-dimensional position for which the reliability is higher.

6. The endoscope system according to claim 5 , wherein

the processor sets the estimated plurality of three-dimensional positions to the three-dimensional position of interest in descending order from the three-dimensional position for which the reliability is highest.

7. The endoscope system according to claim 1 , wherein

the processor selects, among the estimated plurality of three-dimensional positions, the three-dimensional position for which the reliability is judged to be higher than a threshold value as the predetermined three-dimensional position.

8. The endoscope system according to claim 1 , wherein

the processor generates the three-dimensional shape image that is made different in color for each portion depending on the reliability.

9. The endoscope system according to claim 1 , wherein

the processor

estimates a distance between the endoscope and the subject based on the image, and

estimates the three-dimensional position of the target portion based on the estimated distance and a position of the target portion in the image.

10. The endoscope system according to claim 1 , wherein

the processor

acquires position information and posture information of the endoscope, and

estimates the three-dimensional position of the target portion further using the position information and the posture information.

11. The endoscope system according to claim 10 , comprising

a sensor configured to measure a position and a posture of the endoscope, to acquire the position information and the posture information.

12. The endoscope system according to claim 11 , wherein

the predetermined parameter includes at least one of a luminance of the target portion appearing in the image, a light source aperture value for controlling a light amount of illumination light radiated onto the subject, a gain of an image pickup device configured to acquire the image, a position of the target portion appearing in the image relative to an optical center of the image, a hue of the target portion appearing in the image, a movement speed of a distal end of the endoscope, and a reliability for a measurement value by the sensor.

13. The endoscope system according to claim 10 , wherein

the processor estimates a position and a posture of the endoscope based on the image acquired by the endoscope, to acquire the position information and the posture information.

14. An endoscope image processing method comprising:

estimating a three-dimensional position of a target portion within a subject based on a predetermined image acquired by image pickup of an inside of the subject by an endoscope;

setting for the estimated three-dimensional position a reliability corresponding to a predetermined parameter related to an endoscope system determined when the predetermined image is acquired; and

selecting, when a plurality of three-dimensional positions for the target portion exist, a predetermined three-dimensional position among the plurality of three-dimensional positions depending on the set reliability and generating a three-dimensional shape image based on the selected predetermined three-dimensional position.

15. A non-transitory computer readable recording medium storing an endoscope image processing program to be executed by a computer, the computer readable recording medium

estimating, when a processor executes the endoscope image processing program, a three-dimensional position of a target portion within a subject based on a predetermined image acquired by image pickup of an inside of the subject by an endoscope;

setting for the estimated three-dimensional position a reliability corresponding to a predetermined parameter related to the endo scope system determined when the predetermined image is acquired, and

selecting, when a plurality of three-dimensional positions for the target portion exist, a predetermined three-dimensional position among the plurality of three-dimensional positions depending on the set reliability and generating a three-dimensional shape image based on the selected predetermined three-dimensional position.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 4, 2019
From: AKIMOTO, SHUNYA
To: OLYMPUS CORPORATION
Reel/Frame 050903/0338 →
Priority Claims (1)
JP JP2017-117990 · Jun 15, 2017 · national
Continuity (2)
Continuation PCTJP2018013153 · Mar 29, 2018
Related Publication 20200060528A1 · Feb 27, 2020