IP Library › Granted Patent US 12,702,500
Granted Patent B2
US 12,702,500 · App. 18/105,305 · Granted Aug 11, 2026

Controller, endoscope system, and control method

Inventors: Hiroto Ogimoto (Tokyo, JP); Ryota Sasai (Tokyo, JP); Masaru Yanagihara (Tokyo, JP); Hiro Hasegawa (Tokyo, JP); Daichi Kitaguchi (Tokyo, JP); Nobuyoshi Takeshita (Tokyo, JP); Shigehiro Kojima (Tokyo, JP); Yuki Furusawa (Tokyo, JP); Yumi Kinebuchi (Tokyo, JP); Masaaki Ito (Tokyo, JP)
Assignees: OLYMPUS CORPORATION; NATIONAL CANCER CENTER
A61B34/30A61B1/00006A61B1/00009A61B1/0004A61B1/00042A61B1/00045A61B1/00149A61B1/045A61B5/065G06T3/60G06T7/70H04N23/62H04N23/667A61B2034/2055A61B2034/2065A61B2034/301A61B2034/305G06T2207/10012G06T2207/10068G06T2207/30004G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,702,500
App. No.
18/105,305
Filed
Feb 3, 2023
Granted
Aug 11, 2026
Kind
B2
Art Unit
3799
USPC
600/102
Abstract

A controller controls a movement of an endoscope to cause the endoscope to follow a surgical instrument. The controller includes a processor. The processor acquires position information including the position of the surgical instrument, acquires scene information associated with a procedure scene to be observed through the endoscope, determines an offset parameter, which determines the position of the target point with respect to a predetermined fiducial point in the field of view of the endoscope, of a target point on the basis of the scene information, sets the position of the target point with respect to the fiducial point on the basis of the offset parameter, and causes the endoscope to follow the surgical instrument such that the surgical instrument is disposed at the target point, by controlling a movement of the endoscope on the basis of the position of the target point and the position of the surgical instrument.

Claims (31)

1 . A controller comprising:

a processor comprising hardware, the processor being configured to:

acquire position information including a position of a surgical instrument,

acquire scene information related to a procedure of a surgical operation, the scene information being information associated with a procedure scene to be observed through an endoscope,

determine an offset parameter of a target point on a basis of the scene information, the offset parameter being a parameter that determines a position of the target point with respect to a predetermined fiducial point in a field of view of the endoscope,

set the position of the target point with respect to the fiducial point on a basis of the offset parameter, and

control a movement of the endoscope on a basis of the position of the target point and the position of the surgical instrument to three-dimensionally change a positional relationship between a tip of the surgical instrument and the endoscope such that the surgical instrument is disposed at the target point.

2 . The controller according to claim 1 , wherein the processor is configured to:

acquire a moving vector of the surgical instrument as the scene information; and

determine the offset parameter on a basis of the moving vector.

3 . The controller according to claim 2 , wherein the processor is configured to determine the offset parameter that causes a region ahead of the surgical instrument in a moving direction of the surgical instrument to lie at a center of the field of view, on a basis of a direction of the moving vector.

4 . The controller according to claim 1 , wherein the processor is configured to estimate the procedure scene on the basis of the scene information and determines the offset parameter on a basis of the estimated procedure scene.

5 . The controller according to claim 4 , wherein the processor is configured to acquire a type of the surgical instrument as the scene information and estimates the procedure scene on a basis of the type of the surgical instrument.

6 . The controller according to claim 5 , wherein when the procedure scene is estimated to be an ablating scene, the processor is configured to determine the offset parameter that causes a region to be ablated by the surgical instrument to lie at a center of the field of view.

7 . The controller according to claim 5 , wherein when the procedure scene is estimated to be an unfolding scene, the processor is configured to set the offset parameter that causes a region to be unfolded by the surgical instrument to lie at a center of the field of view.

8 . The controller according to claim 4 , wherein the processor is configured to:

acquire an operating state of the surgical instrument as the scene information; and

estimate the procedure scene on a basis of the operating state of the surgical instrument.

9 . The controller according to claim 4 , wherein the processor is configured to:

acquire anatomy information on a biological tissue in the field of view as the scene information; and

estimate the procedure scene on a basis of the anatomy information.

10 . The controller according to claim 1 , wherein the processor is configured to acquire the position information including a three-dimensional position of the surgical instrument.

11 . An endoscope system comprising:

an endoscope;

a moving device that moves the endoscope in a subject; and

the controller according to claim 1 .

12 . The endoscope system according to claim 11 , wherein the endoscope captures an image in the subject and acquires the image in the subject.

13 . The endoscope system according to claim 11 , wherein the moving device includes a robot arm that is connected to the endoscope and controls a position and orientation of the endoscope.

14 . The endoscope system according to claim 13 , wherein the processor is configured to:

calculate the position and orientation of the endoscope such that the target point is disposed at a tip of the surgical instrument, and

control the robot arm on a basis of the calculated position and orientation of the endoscope.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 3, 2023
From: OGIMOTO, HIROTO; SASAI, RYOTA; YANAGIHARA, MASARU; HASEGAWA, HIRO; KITAGUCHI, DAICHI; TAKESHITA, NOBUYOSHI; KOJIMA, SHIGEHIRO; FURUSAWA, YUKI; KINEBUCHI, YUMI; ITO, MASAAKI
To: OLYMPUS CORPORATION; NATIONAL CANCER CENTER
Reel/Frame 062585/0506 →
Continuity (3)
Continuation PCTJP2021033205 · Sep 9, 2021
Provisional Application 63076408 · Sep 10, 2020
Related Publication 20230172675A1 · Jun 8, 2023
References Cited (78)
US 5416513A · Morisaki · 1995 [cited by applicant]
US 6471637B1 · Green et al. · 2002 [cited by applicant]
US 9937013B2 · Frimer et al. · 2018 [cited by applicant]
US 10729502B1 · Wolf et al. · 2020 [cited by applicant]
US 20020156345A1 · Eppler et al. · 2002 [cited by applicant]
US 20020161280A1 · Chatenever et al. · 2002 [cited by applicant]
US 20030055437A1 · Yasunaga · 2003 [cited by applicant]
US 20050020883A1 · Chatenever et al. · 2005 [cited by applicant]
US 20050027167A1 · Chatenever et al. · 2005 [cited by applicant]
US 20050123179A1 · Chen et al. · 2005 [cited by applicant]
US 20070232863A1 · Miyake et al. · 2007 [cited by applicant]
US 20070265502A1 · Minosawa et al. · 2007 [cited by applicant]
US 20130331644A1 · Pandya et al. · 2013 [cited by applicant]
US 20140005475A1 · Song et al. · 2014 [cited by applicant]
US 20140194896A1 · Frimer et al. · 2014 [cited by applicant]
US 20160007827A1 · Frimer et al. · 2016 [cited by applicant]
US 20160007828A1 · Frimer et al. · 2016 [cited by applicant]
US 20160007836A1 · Kikuchi · 2016 [cited by applicant]
US 20160051336A1 · Frimer et al. · 2016 [cited by applicant]
US 20160174955A1 · Frimer et al. · 2016 [cited by applicant]
US 20160270864A1 · Frimer et al. · 2016 [cited by applicant]
US 20160353970A1 · Inoue · 2016 [cited by applicant]
US 20170027654A1 · Frimer et al. · 2017 [cited by applicant]
US 20180140168A1 · Haraguchi · 2018 [cited by applicant]
US 20190000585A1 · Kokubo et al. · 2019 [cited by applicant]
US 20190090728A1 · Fanenbruck et al. · 2019 [cited by applicant]
US 20190117323A1 · Frimer et al. · 2019 [cited by applicant]
US 20190183321A1 · Teranuma · 2019 [cited by applicant]
US 20190209124A1 · Taniguchi · 2019 [cited by applicant]
US 20190365499A1 · Nagao et al. · 2019 [cited by applicant]
US 20200015655A1 · Taguchi · 2020 [cited by applicant]
US 20200113419A1 · Inoue · 2020 [cited by examiner]
US 20200297200A1 · Okada · 2020 [cited by applicant]
US 20200297422A1 · Gocho et al. · 2020 [cited by applicant]
US 20200383736A1 · Frimer et al. · 2020 [cited by applicant]
US 20210212790A1 · Yoshimura et al. · 2021 [cited by applicant]
US 20220079415A1 · Christian et al. · 2022 [cited by applicant]
US 20220192777A1 · Kuroda · 2022 [cited by examiner]
US 20220354347A1 · Nishimura et al. · 2022 [cited by applicant]
US 20220408002A1 · Birkenbach · 2022 [cited by applicant]
EP 1820438A1 · 2007 [cited by applicant]
EP 1854420A1 · 2007 [cited by applicant]
EP 2979610A1 · 2016 [cited by applicant]
JP H05284411A · 1993 [cited by applicant]
JP H05337118A · 1993 [cited by applicant]
JP H06319697A · 1994 [cited by applicant]
JP H0723271A · 1995 [cited by applicant]
JP H07194610A · 1995 [cited by applicant]
JP H0938030A · 1997 [cited by applicant]
JP H09266882A · 1997 [cited by applicant]
JP 2001112704A · 2001 [cited by applicant]
JP 2003088532A · 2003 [cited by applicant]
JP 2003127076A · 2003 [cited by applicant]
JP 2005055756A · 2005 [cited by applicant]
JP 2006229322A · 2006 [cited by applicant]
JP 2007222239A · 2007 [cited by applicant]
JP 2007301378A · 2007 [cited by applicant]
JP 2013192803A · 2013 [cited by applicant]
JP 6161687B2 · 2017 [cited by applicant]
JP 2019162231A · 2019 [cited by applicant]
WO 2005062253A1 · 2005 [cited by applicant]
WO 2018051565A1 · 2018 [cited by applicant]
WO 2018179681A1 · 2018 [cited by applicant]
WO WO2018235255A1 · 2018 [cited by examiner]
WO 2019116592A1 · 2019 [cited by applicant]
WO 2019181149A1 · 2019 [cited by applicant]
WO 2020070883A1 · 2020 [cited by applicant]
US Office Action issued Aug. 22, 2024 in U.S. Appl. No. 18/105,291. [cited by applicant]
Bihlmaier, A., “Learning Dynamic Spatial Relations, The Case of a Knowledge-based Endoscopic Camera Guidance Robot”, pp. 23-102, Springer Link, https://link.springer.com/book/10.1007/978-3-658-14914-7 (2016). [cited by applicant]
International Search Report dated Sep. 28, 2021 issued in PCT/JP2021/027564. [cited by applicant]
International Search Report dated Nov. 22, 2021 issued in PCT/JP2021/033205. [cited by applicant]
International Search Report dated Nov. 22, 2021 issued in PCT/JP2021/033209. [cited by applicant]
International Search Report dated Nov. 16, 2021 issued in PCT/JP2021/033210. [cited by applicant]
Related U.S. Appl. No. 18/105,291, filed Feb. 3, 2023. [cited by applicant]
Related U.S. Appl. No. 18/105,300, filed Feb. 3, 2023. [cited by applicant]
Related U.S. Appl. No. 18/105,314, filed Feb. 3, 2023. [cited by applicant]
US Office Action dated Sep. 10, 2025 received in U.S. Appl. No. 18/105,300. [cited by applicant]
US Office Action dated Sep. 24, 2025 received in U.S. Appl. No. 18/105,314. [cited by applicant]