Granted Patent B2

US 12,342,082 · App. 18/019,911 · Granted Jun 24, 2025

Inventors: Akihiko Nishide (Tokyo, JP); Junko Sugai (Kanagawa, JP)

Assignee: HOYA CORPORATION

H04N23/698G06T5/50G06T5/80H04N13/106

Abstract

Claims (56)

1. A non-transitory computer-readable medium containing a program that causes a computer to execute processing comprising:

acquiring endoscopic images obtained by capturing images of a subject by a plurality of imaging units arranged on an end surface and a peripheral surface of a tubular distal end portion provided in an insertion portion of an endoscope;

generating a composite image obtained by combining a plurality of the captured endoscopic images so as to form a view of an entire circumferential direction including a front visual field and a rear visual field with respect to the distal end portion;

acquiring an insertion distance and a rotation angle of the endoscope inserted into a body of the subject at a time point when the endoscopic images are captured;

outputting the composite image in association with the acquired insertion distance and rotation angle of the endoscope;

generating a plurality of the composite images based on the endoscopic images captured at a plurality of time points;

generating an endoscopic development view in which the insertion distance of the endoscope is set to a first axis and the rotation angle of the endoscope is set to a second axis based on the plurality of the composite images, the generating of the endoscopic development view comprising:

extracting overlapping regions in the plurality of the composite images, and

generating the endoscope deployment view by superimposing the overlapping regions; and

outputting the generated endoscope development view.

2. The non-transitory computer-readable medium containing a program according to claim 1 , wherein

an angle of an optical axis of the imaging unit arranged on the peripheral surface of the distal end portion with respect to an insertion direction of the insertion portion of the endoscope is larger than 90 degrees.

3. The non-transitory computer-readable medium containing a program according to claim 1 , wherein

in a case where the endoscopic images of the subject are captured by an upper endoscopy, an insertion distance of the endoscope is an insertion distance output from a measurement unit provided in an oral cavity portion of the subject, and

in a case where the endoscopic images of the subject are captured by a lower endoscopy, the insertion distance of the endoscope is an insertion distance output from a measurement unit provided in an anal portion of the subject.

4. The non-transitory computer-readable medium containing a program according to claim 1 , wherein

in a case where the endoscopic images of the subject are captured by an upper endoscopy, the rotation angle of the endoscope inserted into the body of the subject is an angle with a vertical direction parallel to a body axis of the subject as a rotation axis, and

in a case where the endoscopic images of the subject are captured by a lower endoscopy, the rotation angle of the endoscope inserted into the body of the subject is an angle with a front-rear direction parallel to a longitudinal direction of a body cavity of the subject as a rotation axis.

5. The non-transitory computer-readable medium containing a program according to claim 1 , the processing further comprising:

specifying a tumor region determined as a tumor in the endoscopic development view using the insertion distance and the rotation angle of the endoscope; and

outputting the endoscope development view with the specified tumor region being highlighted, or displaying comparison with a development view of a virtual endoscopic image by X-ray CT of a corresponding region.

6. The non-transitory computer-readable medium containing a program according to claim 1 , the processing further comprising:

generating a plurality of the composite images based on the endoscopic images captured at a plurality of time points;

generating a stereoscopic image in which a plurality of the endoscopic images captured in different visual field directions is arranged for a same observation site in the body of the subject based on the plurality of the composite images; and

outputting the generated stereoscopic image.

7. The non-transitory computer-readable medium containing a program according to claim 6 , wherein

the processing of generating the stereoscopic image by combining the plurality of the endoscopic images comprises

dividing each of the plurality of the composite images by each visual field direction, and

aligning the respective divided composite images based on the rotation angle of the endoscope and joining the images according to the insertion distance of the endoscope.

8. The non-transitory computer-readable medium containing a program according to claim 6 , the processing further comprising:

extracting a region of interest included in the stereoscopic image; and

outputting diagnosis support information based on the extracted region of interest.

9. An information processing method that causes a computer to execute processing comprising:

acquiring endoscopic images obtained by capturing images of a subject by a plurality of imaging units arranged on an end surface and a peripheral surface of a tubular distal end portion provided in an insertion portion of an endoscope;

generating a composite image obtained by combining a plurality of the captured endoscopic images so as to form a view of an entire circumferential direction including a front visual field and a rear visual field with respect to the distal end portion;

acquiring an insertion distance and a rotation angle of the endoscope inserted into a body of the subject at a time point when the endoscopic images are captured; and

outputting the composite image in association with the acquired insertion distance and rotation angle of the endoscope;

generating a plurality of the composite images based on the endoscopic images captured at a plurality of time points;

generating a stereoscopic image in which a plurality of the endoscopic images captured in different visual field directions is arranged for a same observation site in the body of the subject based on the plurality of the composite images, the generating of the stereoscopic image comprising:

dividing each of the plurality of the composite images by each visual field direction, and

aligning the respective divided composite images based on the rotation angle of the endoscope and joining the images according to the insertion distance of the endoscope; and

outputting the generated stereoscopic image.

10. A non-transitory computer-readable medium containing a program that causes a computer to execute processing comprising:

acquiring endoscopic images obtained by capturing images of a subject by a plurality of imaging units arranged at a distal end portion provided in an insertion portion of an endoscope;

generating a composite image obtained by combining a plurality of the captured endoscopic images so as to form a view of an entire circumferential direction including a front visual field and a rear visual field with respect to the distal end portion; and

outputting the generated composite image, wherein

the processing of generating the composite image comprises:

acquiring reference images obtained by capturing images of a predetermined reference object by a plurality of the imaging units,

deriving an image composite parameter based on the acquired reference images,

generating the composite image using the derived image composite parameter,

performing geometric distortion correction on the acquired endoscopic images, and

combining the plurality of endoscopic images processed in the geometric distortion correction.

11. The non-transitory computer-readable medium containing a program according to claim 10 , wherein

the reference image includes a predetermined pattern having regularity.

12. The non-transitory computer-readable medium containing a program according to claim 10 , wherein

the geometric distortion correction is performed using a curved surface projection method of projecting the acquired endoscopic images onto a continuous curved surface.

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (35)