IP Library › Granted Patent US 12,186,031
Granted Patent B2
US 12,186,031 · App. 18/349,025 · Granted Jan 7, 2025

Graphical user interface for monitoring an image-guided procedure

Inventors: Vincent Duindam (San Francisco, CA); Lauren L. Argo (San Francisco, CA); Cristian Bianchi (Capannori, IT); Christopher R. Carlson (Belmont, CA); Energy Cruse, II (Foster City, CA); Scott S. Ichikawa (Seattle, WA); Aaron B. Tinling (Chimacum, WA); Oliver J. Wagner (Mountain View, CA)
Assignee: INTUITIVE SURGICAL OPERATIONS, INC.
A61B34/20A61B34/10A61B34/25A61B34/35A61B2017/00809A61B2034/105A61B2034/107A61B2034/2051A61B2034/2061A61B2034/2074A61B2034/252A61B2034/254A61B2034/301
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,186,031
App. No.
18/349,025
Granted
Jan 7, 2025
Kind
B2
Abstract

A system and method of monitoring a procedure includes a medical device. The medical device includes an elongate device including a flexible body, a tracking system disposed along at least a portion of the flexible body, and one or more processors communicatively coupled to the tracking system. The one or more processors are configured to receive a route to a target location in an anatomy, determine one or more features of the route based on a first anatomical representation, generate a reduced anatomical representation based on the one or more features of the route, receive real-time position information from the tracking system, associate the real-time position information to the reduced anatomical representation, and dynamically display the reduced anatomical representation with the associated real-time position information.

Claims (53)

1. A medical system comprising:

an elongate device including a flexible body;

a tracking system; and

a control system communicatively coupled to the elongate device and the tracking system, the control system comprising one or more processors, the one or more processors configured to:

output for display a schematic representation of a patient anatomy, wherein the schematic representation includes a plurality of lung segments of the patient anatomy;

receive real-time position information from the tracking system as the elongate device traverses the patient anatomy;

monitor a first amount of real-time position information received from the tracking system as the elongate device traverses a first portion of the patient anatomy; and

update the display of the schematic representation of the patient anatomy by outputting for display a first registration progress indicator of the schematic representation of the patient anatomy, the first registration progress indicator identifying a first lung segment of the plurality of lung segments of the schematic representation of the patient anatomy, the first lung segment of the schematic representation of the patient anatomy corresponding to the first portion of the patient anatomy, wherein the first registration progress indicator is dynamically displayed by gradually transitioning the first lung segment from unfilled to filled as the first amount of real-time position information received from the tracking system increases.

2. The medical system of claim 1 , wherein gradually transitioning the first lung segment from unfilled to filled comprises gradually changing a color of the first lung segment.

3. The medical system of claim 1 , wherein the one or more processors are further configured to:

monitor a second amount of real-time position information received from the tracking system as the elongate device traverses a second portion of the patient anatomy; and

update the display of the schematic representation of the patient anatomy by outputting for display a second registration progress indicator of the schematic representation of the patient anatomy, the second registration progress indicator identifying a second lung segment of the plurality of lung segments of the schematic representation of the patient anatomy, the second lung segment of the schematic representation of the patient anatomy corresponding to the second portion of the patient anatomy.

4. The medical system of claim 3 , wherein the second registration progress indicator is dynamically displayed by gradually transitioning the second lung segment from unfilled to filled as the second amount of real-time position information received from the tracking system increases.

5. The medical system of claim 4 , wherein gradually transitioning the second lung segment from unfilled to filled comprises gradually changing a color of the second lung segment.

6. The medical system of claim 3 , wherein the schematic representation of the patient anatomy, the first registration progress indicator, and the second registration progress indicator are displayed on a display device.

7. The medical system of claim 3 , wherein the one or more processors are further configured to:

determine that a threshold amount of registration data has been received when the first amount of real-time position information received from the tracking system reaches a threshold amount corresponding to the first portion of the patient anatomy and when the second amount of real-time position information received from the tracking system reaches a threshold amount corresponding to the second portion of the patient anatomy; and

based on determining that the threshold amount of registration data has been received, register the elongate device to a model of the patient anatomy.

8. The medical system of claim 3 , wherein the patient anatomy includes a left lung and a right lung, wherein the first portion includes a main stem of the left lung, and wherein the second portion includes a main stem of the right lung.

9. The medical system of claim 1 , wherein the patient anatomy includes a lung, and wherein the first portion includes a main stem of the lung.

10. The medical system of claim 1 , wherein the tracking system includes a position sensor.

11. The medical system of claim 10 , wherein the position sensor comprises one or more electromagnetic sensors.

12. The medical system of claim 10 , wherein the position sensor comprises an optical fiber shape sensor.

13. The medical system of claim 1 , wherein the tracking system includes an insertion sensor.

14. The medical system of claim 13 , wherein the insertion sensor comprises an encoder.

15. The medical system of claim 1 , further comprising a display device, wherein the schematic representation of the patient anatomy is displayed on the display device, and wherein the first registration progress indicator is displayed on the display device.

16. The medical system of claim 1 , further comprising a display device, wherein:

the elongate device comprises an imaging device configured to capture real-time images of the patient anatomy;

the real-time images are displayed on the display device; and

the schematic representation of the patient anatomy is simultaneously displayed on the display device with the real-time images, wherein the schematic representation is displayed in a registration guidance view.

17. The medical system of claim 1 , wherein the one or more processors are further configured to output for display written instructions to direct the elongate device through the patient anatomy, wherein the written instructions are based on the first amount of real-time position information received from the tracking system.

18. The medical system of claim 17 , wherein the written instructions include a suggested area in the patient anatomy the elongate device should traverse so that additional real-time position information can be obtained.

19. A non-transitory machine-readable media storing instructions that, when run by one or more processors, cause the one or more processors to:

display, on a display device, a schematic representation of a patient anatomy, wherein the schematic representation includes a plurality of lung segments of the patient anatomy;

receive real-time position information from a tracking system as a flexible, elongate device traverses the patient anatomy;

monitor a first amount of real-time position information received from the tracking system as the flexible, elongate device traverses a first portion of the patient anatomy; and

in response to the first amount of real-time position information received from the tracking system reaching a threshold amount, update the display of the schematic representation of the patient anatomy by displaying, on the display device, a first registration progress indicator of the schematic representation of the patient anatomy, the first registration progress indicator comprising a first lung segment of the plurality of lung segments of the schematic representation of the patient anatomy, the first lung segment of the schematic representation of the patient anatomy corresponding to the first portion of the patient anatomy.

20. The non-transitory machine-readable media of claim 19 , wherein displaying the first registration progress indicator includes dynamically displaying the first registration progress indicator by gradually transitioning the first lung segment from unfilled to filled as the first amount of real-time position information received from the tracking system increases.

21. The non-transitory machine-readable media of claim 20 , wherein gradually transitioning the first lung segment from unfilled to filled comprises gradually changing a color of the first lung segment.

22. The non-transitory machine-readable media of claim 19 , wherein the instructions further cause the one or more processors to:

monitor a second amount of real-time position information received from the tracking system as the flexible, elongate device traverses a second portion of the patient anatomy; and

in response to the second amount of real-time position information received from the tracking system reaching a threshold amount corresponding to the second portion of the patient anatomy, update the display of the schematic representation of the patient anatomy by displaying, on the display device, a second registration progress indicator of the schematic representation of the patient anatomy, the second registration progress indicator comprising a second lung segment of the plurality of lung segments of the schematic representation of the patient anatomy, the second lung segment of the schematic representation of the patient anatomy corresponding to the second portion of the patient anatomy.

23. The non-transitory machine-readable media of claim 19 , wherein the first lung segment is gradually transitioned from unfilled to filled as an insertion distance of the elongate device within the first portion of the patient anatomy increases.

24. A medical system comprising:

a teleoperated manipulator assembly;

an elongate device coupled to the teleoperated manipulator assembly, the elongate device including a flexible body;

a tracking system; and

a control system communicatively coupled to the elongate device and the tracking system, the control system comprising one or more processors, the one or more processors configured to:

output for display a schematic representation of a patient anatomy;

receive real-time position information from the tracking system as the elongate device traverses the patient anatomy;

monitor a first amount of real-time position information received from the tracking system as the elongate device traverses a first portion of the patient anatomy; and

based on the first amount of real-time position information received from the tracking system reaching a threshold amount, update the display of the schematic representation of the patient anatomy by outputting for display a first registration progress indicator of the schematic representation of the patient anatomy, the first registration progress indicator identifying a first segment of the schematic representation of the patient anatomy, the first segment of the schematic representation of the patient anatomy corresponding to the first portion of the patient anatomy.

25. The medical system of claim 24 , wherein the first registration progress indicator is dynamically displayed by gradually transitioning the first registration progress indicator from unfilled to filled as the first amount of real-time position information received from the tracking system increases.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 16, 2023
From: DUINDAM, VINCENT; ARGO, LAUREN L.; BIANCHI, CRISTIAN; CARLSON, CHRISTOPHER R.; CRUSE, ENERGY, II; ICHIKAWA, SCOTT S.; TINLING, AARON B.; WAGNER, OLIVER J.
To: INTUITIVE SURGICAL OPERATIONS, INC.
Reel/Frame 064610/0811 →
Continuity (3)
Continuation 16499575
Provisional Application 62486879 · Apr 18, 2017
Related Publication 20230346487A1 · Nov 2, 2023
References Cited (129)
US 5638619A · Manwaring et al. · 1997 [cited by applicant]
US 6346940B1 · Fukunaga · 2002 [cited by applicant]
US 6380732B1 · Gilboa · 2002 [cited by applicant]
US 6389187B1 · Greenaway et al. · 2002 [cited by applicant]
US 6432041B1 · Taniguchi et al. · 2002 [cited by applicant]
US 6671538B1 · Ehnholm et al. · 2003 [cited by applicant]
US 7316681B2 · Madhani et al. · 2008 [cited by applicant]
US 7772541B2 · Froggatt et al. · 2010 [cited by applicant]
US 7930065B2 · Larkin et al. · 2011 [cited by applicant]
US 8218846B2 · Trumer et al. · 2012 [cited by applicant]
US 8412307B2 · Willis et al. · 2013 [cited by applicant]
US 8672836B2 · Higgins et al. · 2014 [cited by applicant]
US 8900131B2 · Chopra et al. · 2014 [cited by applicant]
US 9259274B2 · Prisco · 2016 [cited by applicant]
US 9326680B2 · Akimoto et al. · 2016 [cited by applicant]
US 9452276B2 · Duindam et al. · 2016 [cited by applicant]
US 9603668B2 · Weingarten et al. · 2017 [cited by applicant]
US 11612384B2 · Duindam et al. · 2023 [cited by applicant]
US 11819284B2 · Bianchi et al. · 2023 [cited by applicant]
US 20020049375A1 · Strommer et al. · 2002 [cited by applicant]
US 20040082849A1 · Schweikard et al. · 2004 [cited by applicant]
US 20040165810A1 · Fujita · 2004 [cited by applicant]
US 20050182295A1 · Soper et al. · 2005 [cited by applicant]
US 20050261550A1 · Akimoto et al. · 2005 [cited by applicant]
US 20060013523A1 · Childlers et al. · 2006 [cited by applicant]
US 20070065077A1 · Childlers et al. · 2007 [cited by applicant]
US 20070270650A1 · Eno et al. · 2007 [cited by applicant]
US 20070293721A1 · Gilboa · 2007 [cited by applicant]
US 20080183190A1 · Adcox et al. · 2008 [cited by applicant]
US 20080275467A1 · Liao et al. · 2008 [cited by applicant]
US 20090156895A1 · Higgins et al. · 2009 [cited by applicant]
US 20090198104A1 · Sugiyama · 2009 [cited by applicant]
US 20090227861A1 · Ganatra et al. · 2009 [cited by applicant]
US 20090326553A1 · Mustufa et al. · 2009 [cited by applicant]
US 20100076305A1 · Maier-Hein et al. · 2010 [cited by applicant]
US 20100141675A1 · Matsumoto · 2010 [cited by applicant]
US 20100179418A1 · Mueller et al. · 2010 [cited by applicant]
US 20100217117A1 · Glossop et al. · 2010 [cited by applicant]
US 20100249506A1 · Prisco · 2010 [cited by applicant]
US 20100317965A1 · Itkowitz et al. · 2010 [cited by applicant]
US 20110234780A1 · Ito et al. · 2011 [cited by applicant]
US 20110319815A1 · Roelle et al. · 2011 [cited by applicant]
US 20120065481A1 · Hunter et al. · 2012 [cited by applicant]
US 20120089022A1 · House et al. · 2012 [cited by applicant]
US 20120120091A1 · Koudijs et al. · 2012 [cited by applicant]
US 20120289843A1 · Chopra et al. · 2012 [cited by applicant]
US 20130179820A1 · Asami et al. · 2013 [cited by applicant]
US 20130204124A1 · Duindam et al. · 2013 [cited by applicant]
US 20130223702A1 · Holsing et al. · 2013 [cited by applicant]
US 20130281838A1 · Trumer et al. · 2013 [cited by applicant]
US 20130345719A1 · Donhowe · 2013 [cited by examiner]
US 20140142422A1 · Manzke et al. · 2014 [cited by applicant]
US 20140188440A1 · Donhowe · 2014 [cited by examiner]
US 20140211213A1 · Weiss · 2014 [cited by applicant]
US 20140221921A1 · Gilboa et al. · 2014 [cited by applicant]
US 20140350391A1 · Prisco et al. · 2014 [cited by applicant]
US 20150073265A1 · Popovic et al. · 2015 [cited by applicant]
US 20160000302A1 · Brown et al. · 2016 [cited by applicant]
US 20160000356A1 · Brown · 2016 [cited by examiner]
US 20160000517A1 · Kehat et al. · 2016 [cited by applicant]
US 20160070878A1 · Soper · 2016 [cited by examiner]
US 20160073928A1 · Soper et al. · 2016 [cited by applicant]
US 20160183841A1 · Duindam et al. · 2016 [cited by applicant]
US 20160300017A1 · Lee et al. · 2016 [cited by applicant]
US 20160371883A1 · Merkine · 2016 [cited by examiner]
US 20170071504A1 · Wang · 2017 [cited by examiner]
US 20170084027A1 · Mintz et al. · 2017 [cited by applicant]
US 20170151027A1 · Walker et al. · 2017 [cited by applicant]
US 20180177383A1 · Noonan · 2018 [cited by examiner]
US 20180185100A1 · Weinstein · 2018 [cited by examiner]
US 20200030044A1 · Wang et al. · 2020 [cited by applicant]
US 20200054399A1 · Duindam et al. · 2020 [cited by applicant]
US 20200129239A1 · Bianchi et al. · 2020 [cited by applicant]
US 20230200790A1 · Duindam et al. · 2023 [cited by applicant]
CN 101854853A · 2010 [cited by applicant]
CN 104780826A · 2015 [cited by applicant]
CN 105193503A · 2015 [cited by applicant]
CN 105636541A · 2016 [cited by applicant]
CN 106232001A · 2016 [cited by applicant]
EP 1103229A2 · 2001 [cited by applicant]
EP 2085017A1 · 2009 [cited by applicant]
EP 3478161A1 · 2019 [cited by applicant]
JP H08332191A · 1996 [cited by applicant]
JP 2004506466A · 2004 [cited by applicant]
JP 2004513684A · 2004 [cited by applicant]
JP 2008000261A · 2008 [cited by applicant]
JP 2015519987A · 2015 [cited by applicant]
JP 2016030125A · 2016 [cited by applicant]
WO WO9729709A1 · 1997 [cited by applicant]
WO WO0215775A2 · 2002 [cited by applicant]
WO WO2007047782A2 · 2007 [cited by applicant]
WO WO2007146987A2 · 2007 [cited by applicant]
WO WO2008125910A2 · 2008 [cited by applicant]
WO WO2009138871A2 · 2009 [cited by applicant]
WO WO2011043982A1 · 2011 [cited by applicant]
WO WO2012158324A2 · 2012 [cited by applicant]
WO WO2014028394A1 · 2014 [cited by applicant]
WO WO2014139022A1 · 2014 [cited by applicant]
WO WO2014141968A1 · 2014 [cited by applicant]
WO WO2015023665A1 · 2015 [cited by applicant]
WO WO2015164587A2 · 2015 [cited by applicant]
WO WO2016004007A1 · 2016 [cited by applicant]
WO WO2016018646A1 · 2016 [cited by applicant]
WO WO2016018648A1 · 2016 [cited by applicant]
WO WO2016032846A1 · 2016 [cited by applicant]
WO WO2016040080A1 · 2016 [cited by applicant]
WO WO2018005680A1 · 2018 [cited by applicant]
WO WO2018005842A1 · 2018 [cited by applicant]
Extended European Search Report for Application No. EP14836490.4, mailed on Mar. 24, 2017, 9 pages. [cited by applicant]
Extended European Search Report for Application No. EP17821278 mailed on Jan. 23, 2020, 12 pages. [cited by applicant]
Extended European Search Report for Application No. EP17821289 mailed on Feb. 7, 2020, 9 pages. [cited by applicant]
Extended European Search Report for Application No. EP18787852.5 mailed on Mar. 5, 2021, 14 pages. [cited by applicant]
International Preliminary Report on Patentability for Application No. PCT/US2014/050715, mailed on Feb. 25, 2016, 14 pages. [cited by applicant]
International Preliminary Report on Patentability for Application No. PCT/US2018/028190. mailed on Oct. 31, 2019, 11 pages. [cited by applicant]
International Preliminary Reporton Patentability for Application No. PCT/US2017/040095, mailed on Jan. 10, 2019, 8 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2014/050715, mailed on Nov. 13, 2014, 17 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2017/040067, mailed on Aug. 30, 2017, 13 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2017/040095, mailed on Nov. 10, 2017, 11 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2018/028190, mailed on Aug. 3, 2018, 14 pages. [cited by applicant]
Keller S.G., et al., “Equivalent Stress and Strain Distribution in Helical Compression Springs Subjected to Bending,” The Journal of Strain Analysis for Engineering Design, Aug. 2011, vol. 46 (6), pp. 405-415. [cited by applicant]
Office Action for Chinese Application No. CN20188030958, mailed Apr. 14, 2023, 27 pages. [cited by applicant]
Office Action for Chinese Application No. CN20188030958, mailed Aug. 24, 2022, 21 pages. [cited by applicant]
Office Action mailed Apr. 8, 2021 for Chinese Application No. 201780039120 filed Jun. 29, 2017, 28 pages. [cited by applicant]
Partial Supplementary European Search Report for Application No. EP18787852.5 mailed on Dec. 2, 2020, 17 pages. [cited by applicant]
Vertut, J, and Coiffet, P., “Robot Technology: Teleoperation and Robotics Evolution and Development,” English translation, Prentice-Hall, Inc., Inglewood Cliffs, NJ, USA 1986, vol. 3A, 332 pages. [cited by applicant]
Extended European Search Report for Application No. EP23182322.0, mailed on Oct. 2, 2023, 11 pages. [cited by applicant]
Office Action for Chinese Application No. CN20188030958. mailed Sep. 16. 2023. 18 pages. [cited by applicant]
Office Action for Chinese Application No. CN202111558845.9, mailed Dec. 9, 2023, 29 pages. [cited by applicant]
Extended European Search Report for Application No. EP24170572.2, mailed on Jul. 10, 2024, 11 pages. [cited by applicant]