IP Library › Granted Patent US 12,629,218
Granted Patent B2
US 12,629,218 · App. 16/316,747 · Granted May 19, 2026

Systems and methods for displaying an instrument navigator in a teleoperational system

Inventors: Tabish Mustufa (Sunnyvale, CA); Benjamin S. Flamm (Palo Alto, CA); Paul A. Hoover (Bothell, WA)
Assignee: INTUITIVE SURGICAL OPERATIONS, INC.
A61B34/25A61B1/00149A61B1/00193A61B34/20A61B34/30A61B34/35A61B90/00A61B90/37G06T3/60A61B2034/254A61B2034/301A61B2090/365A61B2090/371A61B2090/373
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,629,218
App. No.
16/316,747
Granted
May 19, 2026
Kind
B2
Abstract

A system comprises a teleoperational assembly including a teleoperational manipulator coupled to a plurality of instruments in a surgical environment. The system also comprises a processing unit including one or more processors. The processing unit is configured to display a first synthetic rendering of the plurality of instruments, recognize a triggering event, and change from displaying the first synthetic rendering to displaying a second synthetic rendering of the plurality of instruments in response to recognition of the triggering event.

Claims (34)

1 . A system comprising:

a teleoperational assembly including a teleoperational manipulator coupled to a plurality of instruments in a surgical environment; and

a processing unit including one or more processors, wherein the processing unit is configured to

display a synthetic rendering of the plurality of instruments,

recognize a triggering event comprising a change in an operational state of the teleoperational assembly from a first operational state to a second operational state, wherein movement of a control device causes movement of at least one of the plurality of instruments in only one of the first or second operational states, and

change a perspective associated with displaying the synthetic rendering of the plurality of instruments in response to recognition of the triggering event.

2 . The system of claim 1 wherein the at least one of the plurality of instruments is movable along a central longitudinal axis of the at least one of the plurality of instruments in response to the movement of the control device.

3 . The system of claim 1 wherein the processing unit is further configured to enable an analog control at a user console of the teleoperational assembly and change the perspective associated with displaying the synthetic rendering in response to activation of the enabled analog control.

4 . The system of claim 1 wherein the perspective associated with displaying the synthetic rendering without recognition of the triggering event is distal of the plurality of the instruments and provides a mirrored and laterally inverted view of the plurality of instruments.

5 . The system of claim 4 wherein the perspective associated with displaying the synthetic rendering without recognition of the triggering event is from a perspective plane along a central longitudinal axis through the plurality of instruments.

6 . The system of claim 1 wherein the perspective associated with displaying the synthetic rendering in response to recognition of the triggering event is perpendicular to the perspective of associated with displaying the synthetic rendering without recognition of the triggering event.

7 . The system of claim 1 wherein the perspective associated with displaying the synthetic rendering in response to recognition of the triggering event is from a perspective plane parallel to a central longitudinal axis through the plurality of instruments.

8 . The system of claim 1 wherein at least one of the plurality of instruments is an imaging instrument.

9 . The system of claim 8 wherein the processing unit is further configured to display an image of a field of view of the surgical environment captured by the imaging instrument.

10 . The system of claim 9 wherein the synthetic rendering is displayed in a common display window with the image of the field of view.

11 . The system of claim 9 wherein at least one of the plurality of instruments is not visible in the image of the field of view.

12 . The system of claim 1 wherein the synthetic rendering of the plurality of instruments is generated based on instrument poses of each of the plurality of instruments determined by the teleoperational assembly.

13 . A system comprising:

a teleoperational assembly including a teleoperational manipulator coupled to a plurality of instruments in a surgical environment, the plurality of instruments configured as a bundled unit in an entry guide insertable through a single entry port; and

a processing unit including one or more processors, wherein the processing unit is configured to

generate a synthetic rendering of the plurality of instruments from a perspective distal of a distal end of at least one of the plurality of instruments and along a central longitudinal axis of the bundled unit,

laterally invert the synthetic rendering of the plurality of instruments, and

display the laterally inverted synthetic rendering of the plurality of instruments.

14 . The system of claim 13 wherein at least one of the plurality of instruments is an imaging instrument.

15 . The system of claim 14 wherein the processing unit is further configured to display an image of a field of view of the surgical environment captured by the imaging instrument.

16 . The system of claim 15 wherein the synthetic rendering is displayed in a common display window with the image of the field of view.

17 . The system of claim 15 wherein at least one of the plurality of instruments is not visible in the image of the field of view.

18 . The system of claim 13 wherein the synthetic rendering of the plurality of instruments is generated based on instrument poses of each of the plurality of instruments determined by the teleoperational assembly.

19 . A system comprising:

a teleoperational assembly including a teleoperational manipulator coupled to a plurality of instruments in a surgical environment; and

a processing unit including one or more processors, wherein the processing unit is configured to:

display a synthetic rendering of the plurality of instruments from a perspective of a first virtual camera;

recognize a triggering event comprising a change in an operational state of the teleoperational assembly from a first operational state to a second operational state, wherein movement of a control device causes movement of at least one of the plurality of instruments in only one of the first or second operational states; and

change the displaying of the synthetic rendering of the plurality of instruments to a perspective of a second virtual camera in response to recognition of the triggering event.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2019
From: MUSTUFA, TABISH; FLAMM, BENJAMIN S.; HOOVER, PAUL A.
To: INTUITIVE SURGICAL OPERATIONS, INC.
Reel/Frame 047953/0807 →
Continuity (2)
Provisional Application 62362381 · Jul 14, 2016
Related Publication 20190151032A1 · May 23, 2019
References Cited (33)
US 5471541A · Burtnyk et al. · 1995 [cited by applicant]
US 5836869A · Kudo · 1998 [cited by examiner]
US 6234958B1 · Snoke · 2001 [cited by examiner]
US 6493608B1 · Niemeyer et al. · 2002 [cited by applicant]
US 6636254B1 · Onishi · 2003 [cited by examiner]
US 6671581B2 · Niemeyer et al. · 2003 [cited by applicant]
US 6817974B2 · Cooper et al. · 2004 [cited by applicant]
US 8377045B2 · Schena et al. · 2013 [cited by applicant]
US 9043018B2 · Mohr · 2015 [cited by examiner]
US 20040254454A1 · Kockro · 2004 [cited by applicant]
US 20050203380A1 · Sauer et al. · 2005 [cited by applicant]
US 20060258938A1 · Hoffman · 2006 [cited by examiner]
US 20080118115A1 · Williamson et al. · 2008 [cited by applicant]
US 20080183071A1 · Strommer · 2008 [cited by examiner]
US 20090010508A1 · Inoue · 2009 [cited by examiner]
US 20090192524A1 · Itkowitz et al. · 2009 [cited by applicant]
US 20090326553A1 · Mustufa et al. · 2009 [cited by applicant]
US 20110071473A1 · Rogers et al. · 2011 [cited by applicant]
US 20150038981A1 · Kilroy et al. · 2015 [cited by applicant]
US 20150272694A1 · Charles · 2015 [cited by examiner]
US 20150297300A1 · Gomez et al. · 2015 [cited by applicant]
US 20160220324A1 · Tesar · 2016 [cited by examiner]
US 20170071688A1 · Cohen · 2017 [cited by examiner]
US 20170143442A1 · Tesar · 2017 [cited by examiner]
US 20180324414A1 · Hoffman · 2018 [cited by examiner]
US 20180325604A1 · Atarot · 2018 [cited by examiner]
US 20190142528A1 · Vertikov · 2019 [cited by examiner]
US 20250040789A1 · Nir · 2025 [cited by examiner]
KR 20160010426A · 2016 [cited by applicant]
WO WO2009158164A1 · 2009 [cited by applicant]
International Preliminary Report on Patentability for Application No. PCT/US2017/029400, mailed on Jan. 24, 2019, 9 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2017/029400, mailed on Aug. 7, 2017, 15 pages. [cited by applicant]
Vertut, Jean and Phillipe Coiffet, Robot Technology: Teleoperation and Robotics Evolution and Development, English translation, Prentice-Hall, Inc., Inglewood Cliffs, NJ, USA 1986, vol. 3A, 332 pages. [cited by applicant]