IP Library Granted Patent US 11,612,384
Granted Patent B2
US 11,612,384 · App. 16/307,995 · Granted Mar 28, 2023

Graphical user interface for displaying guidance information in a plurality of modes during an image-guided procedure

Inventors: Vincent Duindam (San Francisco, CA); Federico Barbagli (San Francisco, CA); Cristian Bianchi (Mountain View, CA); Christopher R. Carlson (Belmont, CA); Timothy D. Soper (San Jose, CA)
Assignee: INTUITIVE SURGICAL OPERATIONS, INC.
A61B10/04A61B1/000094A61B34/25A61B90/37A61B1/0005A61B1/2676A61B5/066A61B34/35A61B2010/045A61B2034/2051A61B2034/2061A61B2034/2065A61B2034/301A61B2090/374A61B2090/378A61B2090/3735A61B2090/3762A61M2025/0166
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,612,384
App. No.
16/307,995
Filed
Dec 7, 2018
Granted
Mar 28, 2023
Kind
B2
Examiner
TOWA, RENE T
Art Unit
3791
USPC
600/424
Abstract

A method for displaying guidance information using a graphical user interface during an medical procedure comprises displaying, in a first mode of the graphical user interface, first image data from a perspective corresponding to a distal end of an elongate device. The first image data includes a virtual roadmap. The method also comprises transitioning from the first mode of the graphical user interface to a second mode of the graphical user interface. The transition is based on an occurrence of a triggering condition. The method also comprises displaying, in the second mode of the graphical user interface, second image data from a perspective corresponding to the distal end of the elongate device. The second image data includes a target indicator corresponding to a target location and an alignment indicator corresponding to an expected location of the medical procedure at the target location.

Claims (38)

1. A method for displaying guidance information using a graphical user interface during a medical procedure, the method comprising:

displaying, in a first mode of the graphical user interface, first image data as viewed from a distal end of an elongate device, wherein the first image data includes a virtual roadmap;

transitioning from the first mode of the graphical user interface to a second mode of the graphical user interface wherein the transition is based on an occurrence of a triggering condition, wherein the triggering condition includes detection of a change in presence of a medical instrument in the elongate device based on a change in shape of the elongate device, the change in shape being determined using a shape sensor disposed along a length of the elongate device; and

displaying, in the second mode of the graphical user interface, second image data as viewed from the distal end of the elongate device, wherein the second image data includes a target indicator corresponding to a target location and an alignment indicator corresponding to an expected location of the medical procedure at the target location.

2. The method of claim 1 , further comprising determining a position of the distal end of the elongate device, wherein the triggering condition further includes detecting that the distal end of the elongate device is within a predetermined distance of the target location.

3. The method of claim 1 , wherein at least one of the first image data or the second image data comprises a live camera view.

4. The method of claim 1 , wherein at least one of the first image data or the second image data comprises a virtual image of anatomic passageways.

5. A system for displaying guidance information using a graphical user interface, the system comprising:

a medical instrument comprising an imaging probe;

an elongate device including a flexible body and a distal end; and

one or more processors configured to:

display, in a first mode of the graphical user interface, first image data as viewed from a distal end of the elongate device, wherein the first image data includes a virtual roadmap;

transition from the first mode of the graphical user interface to a second mode of the graphical user interface, wherein the transition is based on an occurrence of a triggering condition that includes a change in presence of the medical instrument within the elongate device, the change in presence of the imaging probe being detected based on a change in video data provided by the imaging probe, the change in the video data being determined using an image processor; and

display, in the second mode of the graphical user interface, second image data as viewed from the distal end of the elongate device, wherein the second image data includes a target indicator corresponding to a target location and an alignment indicator corresponding to an expected location of a medical procedure at the target location;

wherein removal of the imaging probe from the elongate device places the graphical user interface in the second mode or placement of the imaging probe within the elongate device places the graphical user interface in the first mode.

6. The system of claim 5 , wherein the first mode of the graphical user interface is displayed during traversal of the elongate device through an anatomical passageway and wherein the second mode of the graphical user interface is displayed during alignment of a distal end of the elongate device to perform the medical procedure at the target location.

7. The system of claim 5 , wherein the target indicator changes one or more of color, shade, texture, or transparency based on at least one of a distance between the distal end of the elongate device and the target location or based on a deviation between the alignment indicator and the target indicator.

8. The system of claim 5 , wherein the alignment indicator changes one or more of color, shade, texture, or transparency based on a deviation between the alignment indicator and the target indicator.

9. The system of claim 5 , wherein the change in presence of the medical instrument is further detected based on a change in shape of the elongate device, the change in shape being determined using a shape sensor disposed along a length of the elongate device.

10. The system of claim 9 , further comprising a biopsy needle, wherein removal of the biopsy needle from the elongate device places the graphical user interface in the first mode or placement of the biopsy needle within the elongate device places the graphical user interface in the second mode.

11. The system of claim 5 , wherein the one or more processors are further configured to determine a position of the distal end of the elongate device and the triggering condition further includes detecting that the distal end of the elongate device is within a predetermined distance of the target location.

12. The system of claim 11 , further comprising a sensor disposed along a length of the elongate device that determines the position of the distal end of the elongate device, wherein the sensor comprises at least one of an electromagnetic sensor or a fiber optic shape sensor.

13. The system of claim 5 , wherein:

the first image data further includes anatomical features of a patient anatomy;

the virtual roadmap indicates a predetermined route to the target location; and

the one or more processors are further configured to change one or more of color, shade, texture, or transparency of the virtual roadmap based on a deviation between the distal end of the elongate device and the predetermined route.

14. The system of claim 13 , wherein the anatomical features change one or more of color, shade, texture, or transparency based on a distance between a distal end of the elongate device and the target location.

15. The system of claim 13 , wherein the virtual roadmap is displayed as a line superimposed on the anatomical features included in the first image data.

16. The system of claim 13 , wherein the virtual roadmap changes one or more of color, shade, texture, or transparency based on one or more attributes of the anatomical features, the one or more attributes including at least one of a diameter of an anatomical passageway, a bend of the anatomical passageway, a constriction of the anatomical passageway, or a branching of the anatomical passageway.

17. The system of claim 13 , wherein the virtual roadmap changes one or more of color, shade, texture, or transparency based on a distance between the distal end of the elongate device and the target location.

18. The system of claim 13 , wherein displaying the first image data further includes displaying a roadblock indicator.

19. The system of claim 18 , wherein the roadblock indicator is determined based on one or more attributes of the anatomic features, the one or more attributes including a diameter of an anatomical passageway or a bend of the anatomical passageway.

20. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which when executed by one or more processors associated with a guidance display system are adapted to cause the one or more processors to perform a method comprising:

displaying, in a first mode of the guidance display system, first image data as viewed from a distal end of an elongate device, wherein the first image data includes a virtual roadmap;

transitioning from the first mode of the guidance display system to a second mode of the guidance display system wherein the transition is based on an occurrence of a triggering condition, wherein the triggering condition includes a change in presence of a medical instrument in the elongate device based on a change in shape of the elongate device, the change in shape being determined using a shape sensor disposed along a length of the elongate device; and

displaying, in the second mode of the guidance display system, second image data as viewed from the distal end of the elongate device, wherein the second image data includes a target indicator corresponding to a target location and an alignment indicator corresponding to an expected location of a medical procedure at the target location.

21. The non-transitory machine-readable medium of claim 20 , wherein the virtual roadmap comprises one or more arrows indicating a direction to steer the elongate device.

22. The non-transitory machine-readable medium of claim 20 , wherein the virtual roadmap comprises one or more alphanumeric indicators corresponding to at least one of a distance to the target location, a remaining time to reach the target location, or an identity of an anatomical passageway of a set of anatomical passageways.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 7, 2018
From: DUINDAM, VINCENT; BARBAGLI, FEDERICO; BIANCHI, CRISTIAN; CARLSON, CHRISTOPHER R.; SOPER, TIMOTHY D.
To: INTUITIVE SURGICAL OPERATIONS, INC.
Reel/Frame 047703/0248 →
Continuity (3)
Provisional Application 62486879 · Apr 18, 2017
Provisional Application 62357258 · Jun 30, 2016
Related Publication 20200078103A1 · Mar 12, 2020
Cited By (3)
US 12,186,031 US 12,274,514 US 12,721,499