Granted Patent B2

US 12,364,553 · App. 18/373,695 · Granted Jul 22, 2025

Inventors: Tao Zhao (Sunnyvale, CA); Giuseppe Maria Prisco (Sunnyvale, CA); John Ryan Steger (Sunnyvale, CA); David Q. Larkin (Sunnyvale, CA)

Assignee: Intuitive Surgical Operations, Inc.

A61B34/30A61B34/37A61B90/361B25J9/1689B25J9/1697A61B2034/2065

Abstract

Claims (68)

1. A system comprising:

a control means for receiving an input from a user;

a manipulator means configured to support a tool having a tool frame; and

processing means configured to perform a method comprising:

receiving, from an image capturing system with an image frame, one or more images, wherein the tool is visible in the one or more images,

determining an estimated frame transform based on the one or more images, wherein the estimated frame transform is used in defining an unknown frame transform between the image frame and the tool frame,

determining, in response to the input received at the control means, an output movement for the tool based on the estimated frame transform, and

causing movement of the tool based on the output movement.

2. The system of claim 1 , wherein determining the estimated frame transform based on the one or more images comprises:

determining whether to perform a vision measurement on an image of the one or more images; and

in response to determining to perform the vision measurement, using the image of the one or more images in determining the estimated frame transform.

3. The system of claim 1 , wherein determining the output movement for the tool comprises:

temporally smoothing the estimated frame transform.

4. The system of claim 1 , wherein determining the estimated frame transform comprises:

determining an image-to-tool frame relationship between the image frame and the tool frame based on the one or more images; and

computing the estimated frame transform based on the image-to-tool frame relationship.

5. The system of claim 4 , wherein determining the image-to-tool frame relationship further comprises:

identifying pose-indicating points of the tool in the one or more images; and

determining the image-to-tool frame relationship using the pose-indicating points.

6. The system of claim 4 , wherein determining the image-to-tool frame relationship comprises:

determining point pairs derived from a single time instant.

7. The system of claim 4 , wherein determining the image-to-tool frame relationship comprises:

determining point pairs derived from a number of time instants.

8. The system of claim 1 , wherein the method further comprises:

specifying a sequence of frames from the image frame to the tool frame, the sequence of frames including the image frame, the tool frame, and one or more additional frames between the image frame and the tool frame; and

specifying a plurality of frame transforms, each frame transform of the plurality of frame transforms corresponding to a pair of frames of the sequence of frames, the plurality of frame transforms including the unknown frame transform.

9. The system of claim 8 ,

wherein at least one parameter of the unknown frame transform is known, and

wherein determining the estimated frame transform comprises: estimating an unknown parameter of the unknown frame transform.

10. The system of claim 8 ,

wherein the plurality of frame transforms further comprises one or more known frame transforms, and

wherein determining the output movement for the tool comprises: using the one or more known frame transforms in addition to the estimated frame transform.

11. The system of claim 8 , further comprising:

a display means viewable by an operator and associated with an eye frame;

wherein the control means corresponds to a master frame; and

wherein determining the output movement for the tool in response to the input at the control means comprises:

determining a master-to-tool transform by: using the estimated frame transform and one or more known frame transforms of the plurality of frame transforms, and aligning a master-to-eye transform with a tool-to-image transform, wherein the master-to-eye transform transforms points in the master frame to corresponding points in the eye frame, and wherein the tool-to-image transform transforming points in the tool frame to corresponding points in the image frame, and

using the master-to-tool transform to determine the output movement in response to the input.

12. The system of claim 8 , wherein the sequence of frames comprises:

a sequence of frames from the image frame to a fixed frame; and

a second sequence of frames from the fixed frame to the tool frame.

13. The system of claim 8 ,

wherein the plurality of frame transforms comprises a second unknown frame transform; and

wherein the method further comprises: determining a second estimated frame transform for the second unknown frame transform, using the estimated frame transform and one or more known frame transforms of the plurality of frame transforms; and

wherein determining the output movement for the tool in response to the input at the control means comprises: using the second estimated frame transform in addition to the estimated frame transform.

14. A method for operating a system comprising a control means and a manipulator means configured to support a tool having a tool frame, the method comprising:

receiving one or more images captured by an image-capturing system, the image-capturing system having an image frame, wherein the tool is visible in the one or more images;

determining an estimated frame transform based on information in the one or more images, wherein the estimated frame transform is used in defining an unknown frame transform between the image frame and the tool frame;

determining, in response to an input at the control means, an output movement for the tool based on the estimated frame transform; and

causing movement of the tool based on the output movement.

15. The method of claim 14 , wherein determining the estimated frame transform based on the one or more images comprises:

determining whether to perform a vision measurement on an image of the one or more images; and

in response to determining to perform the vision measurement, using the image of the one or more images in determining the estimated frame transform.

16. The method of claim 14 , wherein determining the output movement for the tool comprises:

temporally smoothing the estimated frame transform.

17. The method of claim 14 , wherein determining the estimated frame transform comprises:

determining an image-to-tool frame relationship between the image frame and the tool frame based on the one or more images; and

computing the estimated frame transform based on the image-to-tool frame relationship.

18. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions that, when executed by a processing means associated with a system comprising a control means and a manipulator means configured to support a tool having a tool frame, causes the processing means to perform a method comprising:

receiving one or more images captured by an image-capturing system, the image-capturing system having an image frame, wherein the tool is visible in the one or more images;

determining an estimated frame transform based on information in the one or more images, wherein the estimated frame transform is used in defining an unknown frame transform between the image frame and the tool frame;

determining, in response to an input at the control means, an output movement for the tool based on the estimated frame transform; and

causing movement of the tool based on the output movement.

19. The non-transitory machine-readable medium of claim 18 , wherein determining the estimated frame transform based on the one or more images comprises:

determining whether to perform a vision measurement on an image of the one or more images; and

in response to determining to perform the vision measurement, using the image of the one or more images in determining the estimated frame transform.

20. The non-transitory machine-readable medium of claim 19 , wherein determining the output movement for the tool comprises:

temporally smoothing the estimated frame transform.

Assignments (1)

Continuity (7)

References Cited (70)