IP Library Granted Patent US 12,430,759
Granted Patent B2
US 12,430,759 · App. 17/937,063 · Granted Sep 30, 2025

Image processing device, image processing system, image display method, and image processing program

Inventors: Yasukazu Sakamoto (Hiratsuka, JP); Katsuhiko Shimizu (Fujinomiya, JP); Hiroyuki Ishihara (Tokyo, JP); Clément Jacquet (Sakai, JP); Thomas Henn (Sakai, JP); Stephen Tchen (Sakai, JP); Ryosuke Saga (Osaka, JP)
Assignees: Terumo Kabushiki Kaisha; Rokken Inc.
G06T7/0012G06F3/14G06T7/20G06T7/60G06T7/70G06T15/08G06V10/56G06V10/761G06T2207/10024G06T2207/10072G06T2207/30021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,430,759
App. No.
17/937,063
Granted
Sep 30, 2025
Kind
B2
Abstract

An image processing device is an image processing device configured to cause a display to display three-dimensional data as a three-dimensional image, the three-dimensional data representing a biological tissue. The image processing device includes: a control unit configured to detect a series of positions of a catheter inserted into the biological tissue from data obtained in time series by a sensor observing the surrounding of a lumen of the biological tissue while moving in the lumen, and to switch a display mode between a first mode for displaying a first figure representing the series of positions in the three-dimensional image, and a second mode for displaying a second figure representing one position among the series of positions in the three-dimensional image.

Claims (35)

1. An image processing device configured to cause a display to display three-dimensional data as a three-dimensional image, the three-dimensional data representing a biological tissue, the image processing device comprising:

a control unit configured to:

generate the three-dimensional data based on tomographic data acquired by a sensor that is in a probe inserted into a lumen of the biological tissue, the sensor observing the surrounding of the lumen of the biological tissue while moving in the lumen;

detect a series of positions of a catheter inserted separately from the probe into the biological tissue from the tomographic data obtained in time series by the sensor while the sensor is moving in the lumen; and

switch a display mode between a first mode for displaying a first figure representing the series of positions in the three-dimensional image, and a second mode for displaying a second figure representing one position among the series of positions in the three-dimensional image.

2. The image processing device according to claim 1 , wherein the control unit is configured to hide the first figure when the display mode is switched from the first mode to the second mode.

3. The image processing device according to claim 1 , wherein the control unit is configured to display, as the first figure, a linear or tubular figure connecting the series of positions.

4. The image processing device according to claim 1 , wherein the control unit is configured to display, as the second figure, a spherical figure representing the one position, and display at least one concentric circle or concentric sphere surrounding the spherical figure.

5. The image processing device according to claim 1 , wherein the control unit is configured to display, as the second figure, a figure in which a thickness dimension on a side near an end of the catheter is larger than a thickness dimension on an opposite side.

6. The image processing device according to claim 5 , wherein the control unit is configured to display, as the second figure, a bullet-shaped figure representing the one position.

7. The image processing device according to claim 1 , wherein when displaying a figure representing a position other than a distal end of the catheter as the second figure, the control unit is configured to display the second figure in a color different from that when displaying a figure representing a position of the distal end of the catheter as the second figure.

8. The image processing device according to claim 1 , wherein the control unit is configured to switch the display mode according to a change in a moving state of the sensor.

9. The image processing device according to claim 8 , wherein the change in the moving state of the sensor includes a case in which a moving speed of the sensor exceeds a threshold value.

10. The image processing device according to claim 9 , wherein when the moving speed of the sensor exceeds the threshold value, the control unit is configured to switch the display mode according to whether the moving speed of the sensor is equal to or less than an upper limit value.

11. The image processing device according to claim 1 , wherein the control unit is configured to switch the display mode according to whether the catheter is present in an image plane captured by the sensor.

12. The image processing device according to claim 11 , wherein when the catheter is present in the image plane, the control unit is configured to switch the display mode according to a magnitude of a movement of the catheter in the image plane.

13. The image processing device according to claim 11 , wherein when the catheter is not present in the image plane, the control unit is configured to switch the display mode according to an elapsed time from a time point at which a last position among the series of positions is detected.

14. The image processing device according to claim 1 , wherein in the second mode, the control unit is configured to adjust at least one of a size or a color of the second figure according to a distance from an inner wall surface of the biological tissue.

15. The image processing device according to claim 1 , wherein in the second mode, the control unit is configured to adjust at least one of a size or a color of the second figure according to presence or absence of contact with the inner wall surface of the biological tissue.

16. The image processing device according to claim 1 , wherein in the second mode, the control unit is further configured to display a shadow of the second figure.

17. The image processing device according to claim 1 , wherein in the second mode, the control unit is configured to adjust an update frequency of the second figure associated with an update of the data.

18. An image processing system, comprising:

the sensor configured to acquire the tomographic data of the biological tissue while moving in the lumen of the biological tissue; and

the image processing device according to claim 1 , the image processing device configured to generate the three-dimensional data based on the tomographic data acquired by the sensor.

19. The image processing system according to claim 18 , further comprising:

the display.

20. An image display method for causing a display to display three-dimensional data as a three-dimensional image, the three-dimensional data representing a biological tissue, the image display method comprising:

generating, by a processor, the three-dimensional data based on tomographic data acquired by a sensor that is in a probe inserted into a lumen of the biological tissue, the sensor observing the surrounding of the lumen of the biological tissue while moving in the lumen;

detecting, by the processor, a series of positions of a catheter inserted separately from the probe into the biological tissue from the tomographic data obtained in time series by the sensor while the sensor is moving in the lumen; and

switching, by the processor, a display mode between a first mode for displaying a first figure representing the series of positions in the three-dimensional image, and a second mode for displaying a second figure representing one position among the series of positions in the three-dimensional image.

21. A non-transitory computer-readable medium (CRM) storing computer program code executed by a computer processor that executes an imaging process comprising:

generating three-dimensional data representing a biological tissue based on tomographic data acquired by a sensor that is in a probe inserted into a lumen of the biological tissue, the sensor observing the surrounding of the lumen of the biological tissue while moving in the lumen;

displaying, on a display, the three-dimensional data as a three-dimensional image;

detecting a series of positions of a catheter inserted separately from the probe into the biological tissue from the tomographic data obtained in time series by the sensor while the sensor is moving in the lumen; and

switching a display mode between a first mode for displaying a first figure representing the series of positions in the three-dimensional image, and a second mode for displaying a second figure representing one position among the series of positions in the three-dimensional image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 24, 2023
From: SAKAMOTO, YASUKAZU; SHIMIZU, KATSUHIKO; ISHIHARA, HIROYUKI; JACQUET, CLÉMENT; HENN, THOMAS; TCHEN, STEPHEN; SAGA, RYOSUKE
To: TERUMO KABUSHIKI KAISHA; ROKKEN INC.
Reel/Frame 065319/0740 →
Priority Claims (1)
JP 2020-061800 · Mar 31, 2020 · national
Continuity (2)
Continuation PCTJP2021011534 · Mar 19, 2021
Related Publication 20230025720A1 · Jan 26, 2023
References Cited (40)
US 6251072B1 · Ladak et al. · 2001 [cited by applicant]
US 6385332B1 · Zahalka et al. · 2002 [cited by applicant]
US 11089351B2 · Yoshizawa · 2021 [cited by examiner]
US 11806094B2 · Sheiner · 2023 [cited by examiner]
US 11842456B2 · Gluhovsky · 2023 [cited by examiner]
US 20040097805A1 · Verard · 2004 [cited by examiner]
US 20050107688A1 · Strommer · 2005 [cited by examiner]
US 20050288577A1 · Weese · 2005 [cited by applicant]
US 20070038081A1 · Eck et al. · 2007 [cited by applicant]
US 20070123771A1 · Redel · 2007 [cited by examiner]
US 20080247506A1 · Maschke · 2008 [cited by examiner]
US 20100215238A1 · Lu et al. · 2010 [cited by applicant]
US 20110166418A1 · Aoyagi et al. · 2011 [cited by applicant]
US 20140135618A1 · Abe et al. · 2014 [cited by applicant]
US 20150223670A1 · Fujita et al. · 2015 [cited by applicant]
US 20150320979A1 · Fearnot et al. · 2015 [cited by applicant]
US 20160228089A1 · Jamello · 2016 [cited by examiner]
US 20170024910A1 · Griffin et al. · 2017 [cited by applicant]
US 20180228554A1 · Strommer · 2018 [cited by examiner]
US 20200129142A1 · Chao · 2020 [cited by examiner]
CN 102119848A · 2011 [cited by applicant]
CN 103717135A · 2014 [cited by applicant]
JP H10137238A · 1998 [cited by applicant]
JP 2006508731A · 2006 [cited by applicant]
JP 2007503906A · 2007 [cited by applicant]
JP 2008113699A · 2008 [cited by applicant]
JP 2011193995A · 2011 [cited by applicant]
JP 2014083289A · 2014 [cited by applicant]
JP 2016516466A · 2016 [cited by applicant]
JP 2017514617A · 2017 [cited by applicant]
JP 2018520839A · 2018 [cited by applicant]
WO WO2005112750A1 · 2005 [cited by examiner]
WO 2014175853A1 · 2014 [cited by applicant]
WO 2015044983A1 · 2015 [cited by applicant]
WO WO2018081012A1 · 2018 [cited by examiner]
WO WO2018200738A1 · 2018 [cited by examiner]
WO 2019239647A1 · 2019 [cited by applicant]
International Search Report (PCT/ISA/210) with English translation and Written Opinion (PCT/ISA/237) mailed on Jun. 8, 2021, by the Japanese Patent Office as the International Searching Authority for International Appli… [cited by applicant]
International Preliminary Report on Patentability (Form PCT/IB/373) and an English Translations of the Written Opinion of the International Searching Authority (Form PCT/ISA/237) issued Sep. 29, 2022, by the Internation… [cited by applicant]
Office Action/Search Report (The First Office Action) issued on May 30, 2025, in corresponding Chinese Patent Application No. 202180026956.7 and English translation of the Office Action/Search Report. (14 pages). [cited by applicant]