Granted Patent B2

US 12,266,040 · App. 18/437,072 · Granted Apr 1, 2025

Inventors: Brandon D. Itkowitz (San Jose, CA); Simon P. DiMaio (San Carlos, CA); Daniel J. Halabe (Los Altos, CA); Christopher J. Hasser (Los Altos, CA); Brian David Hoffman (Mountain View, CA); David Q. Larkin (Menlo Park, CA); Catherine J. Mohr (Mountain View, CA); Paul W. Mohr (Mountain View, CA); Tao Zhao (Sunnyvale, CA); Wenyi Zhao (Weston, FL)

Assignee: INTUITIVE SURGICAL OPERATIONS, INC.

G06T11/60A61B34/30A61B34/37A61B90/36B25J9/1666B25J9/1671B25J9/1692G05B19/4202G06T11/40A61B34/20A61B2034/2061A61B34/25A61B90/361A61B2090/371G05B2219/36432G05B2219/39083G05B2219/39096G05B2219/39449G05B2219/40607G05B2219/45117G05B2219/45123Y02P90/02

Abstract

Claims (37)

1. A system comprising:

a first robotic arm operatively coupleable to a first tool, the first tool comprising a working end;

a first input device communicatively coupled to the first robotic arm, the first input device being manipulatable by a first operator; and

a processor configured to:

cause an image of a work site, captured by an image capture device from a perspective of an image reference frame, to be displayed on a display, the image of the work site including an image of the working end of the first tool;

determine a position of the working end of the first tool in the image of the work site; and

render a first tool information overlay at the position of the working end of the first tool in the image of the work site, wherein the first tool information overlay visually indicates an identity of the first input device.

2. The system of claim 1 , wherein the first input device is a left input device, and wherein the first tool information overlay is a letter “L”.

3. The system of claim 1 , wherein the first input device is a right input device, and wherein the first tool information overlay is a letter “R”.

4. The system of claim 1 , wherein the first tool information overlay further visually indicates an identity of the first operator.

5. The system of claim 4 , wherein the first tool information overlay includes a symbol or a color associated with the first operator.

6. The system of claim 1 , further comprising a second input device communicatively coupled to the first robotic arm, the second input device being manipulatable by a second operator.

7. The system of claim 6 , wherein when the first input device is operatively associated with the first robotic arm, the first tool information overlay is displayed in a first color associated with the first operator.

8. The system of claim 7 , wherein when the second input device is operatively associated with the first robotic arm, the first tool information overlay is displayed in a second color associated with the second operator.

9. The system of claim 8 , wherein the processor is further configured to:

change a display of the first tool information overlay from the first color to the second color when the first robotic arm changes from being operatively associated with the first input device to being operatively associated with the second input device.

10. The system of claim 1 , further comprising:

a second robotic arm operatively coupleable to a second tool, the second tool comprising a working end; and

a second input device communicatively coupled to the second robotic arm, the second input device being manipulatable by the first operator,

wherein the image of the work site further includes an image of the working end of the second tool, and

wherein the processor is further configured to:

determine a position of the working end of the second tool in the image of the work site; and

render a second tool information overlay at the position of the working end of the second tool in the image of the work site, wherein the second tool information overlay visually indicates an identity of the second input device.

11. The system of claim 1 , wherein the first tool information overlay is displayed over an end effector of the working end of the first tool.

12. The system of claim 1 , wherein the first tool information overlay is displayed over a portion of the working end of the first tool proximal of an end effector.

13. A method comprising:

displaying an image of a work site to be displayed on a display, the image captured by an image capture device from a perspective of an image reference frame, the image of the work site including an image of a working end of a first tool operatively coupled to a first robotic arm;

determining a position of the working end of the first tool in the image of the work site; and

rendering a first tool information overlay at the position of the working end of the first tool in the image of the work site, wherein the first tool information overlay visually indicates an identity of a first input device manipulatable by a first operator, the first input device communicatively coupled to the first robotic arm.

14. The method of claim 13 , wherein the first input device is a left input device, and wherein the first tool information overlay is a letter “L”.

15. The method of claim 13 , wherein the first input device is a right input device, and wherein the first tool information overlay is a letter “R”.

16. The method of claim 13 , wherein the first tool information overlay further visually indicates an identity of the first operator.

17. The method of claim 13 , wherein the first tool information overlay further visually indicates an identity of a second input device manipulatable by a second operator, the second input device communicatively coupled to the first robotic arm.

18. The method of claim 17 , wherein when the first input device is operatively associated with the first robotic arm, the first tool information overlay is displayed in a first color associated with the first operator.

19. The method of claim 18 , wherein when the second input device is operatively associated with the first robotic arm, the first tool information overlay is displayed in a second color associated with the second operator.

20. The method of claim 19 , further comprising:

changing a display of the first tool information overlay from the first color to the second color when the first robotic arm changes from being operatively associated with the first input device to being operatively associated with the second input device.

Assignments (2)

Continuity (6)

References Cited (400)