IP Library › Granted Patent US 12,527,635
Granted Patent B2
US 12,527,635 · App. 18/150,105 · Granted Jan 20, 2026

Graphical user interface for planning a procedure

Inventors: Bai Wang (Palo Alto, CA); Federico Barbagli (San Francisco, CA); Energy Cruse, II (Foster City, CA); Scott S. Ichikawa (Seattle, WA); Pechin Chien Pau Lo (Santa Clara, CA); Oscar G. Salazar (Sunnyvale, CA); Aaron B. Tinling (Chimacum, WA); Oliver J. Wagner (Mountain View, CA); Tao Zhao (Sunnyvale, CA)
Assignee: INTUITIVE SURGICAL OPERATIONS, INC.
A61B34/25A61B34/10G06F3/04815G06F3/04845G06F3/04883A61B2034/105A61B2034/107
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,527,635
App. No.
18/150,105
Filed
Jan 4, 2023
Granted
Jan 20, 2026
Kind
B2
Examiner
KELLS, ASHER
Art Unit
2171
USPC
715/764
Abstract

A planning workstation includes a display system and a user input device and is configured to display a plurality of interactive windows for a user to view a plan for a medical procedure. Each of the plurality of interactive windows displays a different rendering of a model of anatomical passageways. The planning workstation is further configured to display a path through the anatomical passageways to a target of the medical procedure, display a virtual image of an instrument within the anatomical passageways, display a control point corresponding to a distal end of the instrument, receive a user input via the user input device, identify a position of the control point based at least on the user input, and, in response to receiving the user input, dynamically update a position of the instrument in at least one of the interactive windows to match the position of the control point.

Claims (43)

1 . A planning workstation comprising:

a display system; and

a user input device;

wherein the planning workstation is configured to, during pre-operative planning of a procedure:

display a plurality of interactive windows for a user to view a plan for a medical procedure, wherein each of the plurality of interactive windows displays a different rendering of a model of anatomical passageways;

display a path through the anatomical passageways to a target of the medical procedure in a global anatomical model view displaying a rendering of the model of the anatomical passageways from an external perspective;

display a virtual image corresponding to a position of an instrument within the anatomical passageways;

display a control point corresponding to a distal end of the instrument in at least one of the plurality of interactive windows;

receive, via the user input device, a user input adjusting a position of the control point;

and

in response to receiving the user input, dynamically update the position of the instrument in at least one of the plurality of interactive windows to match the adjusted position of the control point.

2 . The planning workstation of claim 1 , wherein the user input device includes a touchscreen of the display system.

3 . The planning workstation of claim 1 , wherein the control point is displayed in a first interactive window of the plurality of interactive windows, the first interactive window including a reduced anatomical model view, the user input being received in relation to the reduced anatomical model view.

4 . The planning workstation of claim 3 , wherein the reduced anatomical model view comprises a simplified overview of a planned route of the instrument through the anatomical passageways including key anatomical features.

5 . The planning workstation of claim 3 , wherein the position of the instrument is dynamically updated in a second interactive window of the plurality of interactive windows, the second interactive window including a simulated live endoscopic view.

6 . The planning workstation of claim 3 , wherein the position of the instrument is dynamically updated in a second interactive window of the plurality of interactive windows, the second interactive window including a virtual endoscopic view.

7 . The planning workstation of claim 6 , wherein the virtual endoscopic view comprises at least one directional cue guiding the user toward the target.

8 . The planning workstation of claim 3 , wherein the position of the instrument is dynamically updated in a second interactive window of the plurality of interactive windows, the second interactive window including the global anatomical model view.

9 . The planning workstation of claim 8 , wherein dynamically updating the position of the instrument in the second interactive window includes updating a shape of the instrument displayed in the second interactive window.

10 . A method of planning a medical procedure, the method performed during pre-operative planning of a procedure and comprising:

displaying a plurality of interactive windows on a display system for a user to view a plan for a medical procedure, wherein each of the plurality of interactive windows displays a different rendering of a model of anatomical passageways;

displaying a path through the anatomical passageways to a target of the medical procedure in a global anatomical model view displaying a rendering of the model of the anatomical passageways from an external perspective;

displaying a virtual image corresponding to a position of an instrument within the anatomical passageways;

displaying a control point corresponding to a distal end of the instrument in at least one of the plurality of interactive windows;

receiving, via a user input device, a user input adjusting a position of the control point; and

in response to receiving the user input, dynamically updating the position of the instrument in at least one of the plurality of interactive windows to match the adjusted position of the control point.

11 . The method of claim 10 , wherein the user input device includes a touchscreen of the display system.

12 . The method of claim 10 , wherein the control point is displayed in a first interactive window of the plurality of interactive windows, the first interactive window including a reduced anatomical model view, the user input being received in relation to the reduced anatomical model view.

13 . The method of claim 12 , wherein the reduced anatomical model view comprises a simplified overview of a planned route of the instrument through the anatomical passageways including key anatomical features.

14 . The method of claim 12 , wherein the position of the instrument is dynamically updated in a second interactive window of the plurality of interactive windows, the second interactive window including a simulated live endoscopic view.

15 . The method of claim 12 , wherein the position of the instrument is dynamically updated in a second interactive window of the plurality of interactive windows, the second interactive window including a virtual endoscopic view.

16 . The method of claim 15 , wherein the virtual endoscopic view comprises at least one directional cue guiding the user toward the target.

17 . The method of claim 12 , wherein the position of the instrument is dynamically updated in a second interactive window of the plurality of interactive windows, the second interactive window including the global anatomical model view.

18 . The method of claim 17 , wherein dynamically updating the position of the instrument in the second interactive window includes updating a shape of the instrument displayed in the second interactive window.

19 . A non-transitory machine readable medium comprising a plurality of machine readable instructions which when executed by one or more processors associated with a planning workstation are adapted to cause the one or more processors to perform a method of pre-operative planning of a procedure comprising:

providing a plurality of interactive windows for a user to view a plan for a medical procedure, wherein each of the plurality of interactive windows displays a different rendering of a model of anatomical passageways;

displaying a path through the anatomical passageways to a target of the medical procedure in a global anatomical model view displaying a rendering of the model of the anatomical passageways from an external perspective;

displaying a virtual image corresponding to a position of an instrument within the anatomical passageways;

displaying a control point corresponding to a distal end of the instrument in at least one of the plurality of interactive windows;

receiving a user input adjusting a position of the control point;

and

in response to receiving the user input, dynamically updating the position of the instrument in at least one of the plurality of interactive windows to match the adjusted position of the control point.

20 . The non-transitory machine readable medium of claim 19 , wherein the control point is displayed in a first interactive window of the plurality of interactive windows, the first interactive window including a reduced anatomical model view, the user input being received in relation to the reduced anatomical model view.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2023
From: WANG, BAI; BARBAGLI, FEDERICO; CRUSE, ENERGY, II; ICHIKAWA, SCOTT S.; LO, PECHIN CHIEN PAU; SALAZAR, OSCAR G.; TINLING, AARON B.; WAGNER, OLIVER J.; ZHAO, TAO
To: INTUITIVE SURGICAL OPERATIONS, INC.
Reel/Frame 062283/0546 →
Continuity (4)
Continuation 16496034
Provisional Application 62578189 · Oct 27, 2017
Provisional Application 62486896 · Apr 18, 2017
Related Publication 20230145309A1 · May 11, 2023
References Cited (108)
US 6203493B1 · Ben-Haim · 2001 [cited by applicant]
US 6380732B1 · Gilboa · 2002 [cited by applicant]
US 6389187B1 · Greenaway et al. · 2002 [cited by applicant]
US 7316681B2 · Madhani et al. · 2008 [cited by applicant]
US 7772541B2 · Froggatt et al. · 2010 [cited by applicant]
US 8900131B2 · Chopra et al. · 2014 [cited by applicant]
US 9259274B2 · Prisco · 2016 [cited by applicant]
US 9259290B2 · Jenkins et al. · 2016 [cited by applicant]
US 9452276B2 · Duindam et al. · 2016 [cited by applicant]
US 9639666B2 · Baker · 2017 [cited by applicant]
US 9763745B2 · Karmarkar et al. · 2017 [cited by applicant]
US 10373719B2 · Soper et al. · 2019 [cited by applicant]
US 10610311B2 · Herbelin et al. · 2020 [cited by applicant]
US 11412951B2 · Piron et al. · 2022 [cited by applicant]
US 20040034302A1 · Abovitz · 2004 [cited by examiner]
US 20040249267A1 · Gilboa et al. · 2004 [cited by applicant]
US 20050107679A1 · Geiger et al. · 2005 [cited by applicant]
US 20050182295A1 · Soper et al. · 2005 [cited by applicant]
US 20060013523A1 · Childlers et al. · 2006 [cited by applicant]
US 20060058647A1 · Strommer et al. · 2006 [cited by applicant]
US 20070167738A1 · Timinger et al. · 2007 [cited by applicant]
US 20070197896A1 · Moll · 2007 [cited by examiner]
US 20070249911A1 · Simon · 2007 [cited by applicant]
US 20080081982A1 · Simon et al. · 2008 [cited by applicant]
US 20080097187A1 · Gielen et al. · 2008 [cited by applicant]
US 20080118135A1 · Averbuch et al. · 2008 [cited by applicant]
US 20080264430A2 · Roschak · 2008 [cited by applicant]
US 20090112082A1 · Piferi et al. · 2009 [cited by applicant]
US 20090171184A1 · Jenkins et al. · 2009 [cited by applicant]
US 20090177077A1 · Piferi et al. · 2009 [cited by applicant]
US 20090248045A1 · Trovato · 2009 [cited by applicant]
US 20100054525A1 · Gong et al. · 2010 [cited by applicant]
US 20100082692A1 · Akinyemi et al. · 2010 [cited by applicant]
US 20100121316A1 · Weese et al. · 2010 [cited by applicant]
US 20100198402A1 · Greer · 2010 [cited by examiner]
US 20100204567A1 · Narouze · 2010 [cited by applicant]
US 20100204568A1 · Narouze · 2010 [cited by applicant]
US 20100275145A1 · Nijlunsing et al. · 2010 [cited by applicant]
US 20100310146A1 · Higgins et al. · 2010 [cited by applicant]
US 20110075920A1 · Wu et al. · 2011 [cited by applicant]
US 20120143198A1 · Boyer et al. · 2012 [cited by applicant]
US 20120203065A1 · Higgins et al. · 2012 [cited by applicant]
US 20130166011A1 · Strommer · 2013 [cited by applicant]
US 20130204124A1 · Duindam · 2013 [cited by applicant]
US 20130317351A1 · Case et al. · 2013 [cited by applicant]
US 20130338437A1 · Abuzaina · 2013 [cited by applicant]
US 20140187949A1 · Zhao et al. · 2014 [cited by applicant]
US 20140188440A1 · Donhowe et al. · 2014 [cited by applicant]
US 20140281961A1 · Baker · 2014 [cited by applicant]
US 20140343416A1 · Panescu et al. · 2014 [cited by applicant]
US 20150049081A1 · Coffey et al. · 2015 [cited by applicant]
US 20150182297A1 · Sandhu · 2015 [cited by examiner]
US 20150257847A1 · Higgins et al. · 2015 [cited by applicant]
US 20160000302A1 · Brown et al. · 2016 [cited by applicant]
US 20160000414A1 · Brown et al. · 2016 [cited by applicant]
US 20160018647A1 · Osterhout et al. · 2016 [cited by applicant]
US 20160070436A1 · Thomas et al. · 2016 [cited by applicant]
US 20160092722A1 · Soenksen et al. · 2016 [cited by applicant]
US 20160155232A1 · Sela et al. · 2016 [cited by applicant]
US 20160166335A1 · Roger et al. · 2016 [cited by applicant]
US 20160183841A1 · Duindam et al. · 2016 [cited by applicant]
US 20160192991A1 · Vahala et al. · 2016 [cited by applicant]
US 20160235492A1 · Morard et al. · 2016 [cited by applicant]
US 20160371883A1 · Merkine et al. · 2016 [cited by applicant]
US 20170020628A1 · Averbuch · 2017 [cited by applicant]
US 20170035517A1 · Geri et al. · 2017 [cited by applicant]
US 20170065349A1 · Ourselin et al. · 2017 [cited by applicant]
US 20170072222A1 · Siversson · 2017 [cited by applicant]
US 20170084027A1 · Mintz et al. · 2017 [cited by applicant]
US 20170189118A1 · Chopra et al. · 2017 [cited by applicant]
US 20170193160A1 · Long et al. · 2017 [cited by applicant]
US 20170258526A1 · Lang · 2017 [cited by applicant]
US 20170265952A1 · Donhowe et al. · 2017 [cited by applicant]
US 20170312032A1 · Amanatullah et al. · 2017 [cited by applicant]
US 20180041685A1 · Carbonell et al. · 2018 [cited by applicant]
US 20180055574A1 · Krimsky · 2018 [cited by applicant]
US 20180098716A1 · Cohen et al. · 2018 [cited by applicant]
US 20180132946A1 · Kao et al. · 2018 [cited by applicant]
US 20180161113A1 · Lee et al. · 2018 [cited by applicant]
US 20180199999A1 · Syverson et al. · 2018 [cited by applicant]
US 20180318009A1 · Sohlden · 2018 [cited by examiner]
US 20190192096A1 · Wu et al. · 2019 [cited by applicant]
US 20190350659A1 · Wang et al. · 2019 [cited by applicant]
US 20200022649A1 · Rodriguez et al. · 2020 [cited by applicant]
US 20200030044A1 · Wang et al. · 2020 [cited by applicant]
US 20200037916A1 · Partanen et al. · 2020 [cited by applicant]
US 20200051274A1 · Siemionow et al. · 2020 [cited by applicant]
US 20200054399A1 · Duindam et al. · 2020 [cited by applicant]
US 20200078103A1 · Duindam et al. · 2020 [cited by applicant]
US 20200100851A1 · Marcuk · 2020 [cited by applicant]
US 20200129239A1 · Bianchi et al. · 2020 [cited by applicant]
US 20200297442A1 · Adebar et al. · 2020 [cited by applicant]
US 20210177299A1 · Prisco et al. · 2021 [cited by applicant]
CN 1907233A · 2007 [cited by applicant]
CN 103957834A · 2014 [cited by applicant]
CN 104050313A · 2014 [cited by applicant]
WO WO2015023665A1 · 2015 [cited by applicant]
WO WO2015119935A1 · 2015 [cited by applicant]
WO WO2016018648A1 · 2016 [cited by applicant]
Office Action for Chinese Application No. CN201880033554.8, mailed Feb. 25, 2023, 22 pages. [cited by applicant]
Extended European Search Report for Application No. EP18787521.6 mailed on Mar. 24, 2021, 12 pages. [cited by applicant]
International Preliminary Report on Patentability for Application No. PCT/US2018/028199, mailed on Oct. 31, 2019, 11 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2018/028199, mailed on Aug. 6, 2018, 12 pages. [cited by applicant]
Mori K., et al., “Automated Anatomical Labeling of the Bronchial Branch and its Application to the Virtual Bronchoscopy System,” IEEE Transactions on Medical Imaging, Feb. 2000, vol. 19 (2), pp. 103-114. [cited by applicant]
Partial Supplementary European Search Report for Application No. 18787521.6 mailed on Dec. 22, 2020, 14 pages. [cited by applicant]
Vertut, J, and Coiffet, P., “Robot Technology: Teleoperation and Robotics Evolution and Development,” English translation, Prentice-Hall, Inc., Inglewood Cliffs, NJ, USA 1986, vol. 3A, 332 pages. [cited by applicant]
Office Action for Chinese Application No. CN201880033554.8, mailed Sep. 21, 2023, 21 pages. [cited by applicant]
Extended European Search Report for Application No. EP25160185.2, mailed on May 27, 2025, 12 pages. [cited by applicant]