Estimation of a position and orientation of a frame used in controlling movement of a tool

1. A system comprising:

a control device;

a manipulator configured to support a tool having a tool frame; and

at least one processor coupled to the control device and the manipulator, the at least one processor configured to perform a method comprising:

receiving one or more images captured by an image-capturing system, the image-capturing system having an image frame, wherein the tool is visible in the one or more images,

determining an estimated frame transform based on the one or more images, wherein the estimated frame transform is used in defining an unknown frame transform between the image frame and the tool frame,

determining, in response to an input received at the control device, an output movement for the tool based on the estimated frame transform, and

causing movement of the tool based on the output movement.

2. The system of claim 1 , wherein determining the estimated frame transform based on the one or more images comprises:

determining whether to perform a vision measurement on an image of the one or more images; and

in response to determining to perform the vision measurement, using the image of the one or more images in determining the estimated frame transform.

3. The system of claim 1 , wherein determining the output movement for the tool comprises:

temporally smoothing the estimated frame transform.

4. The system of claim 1 , wherein determining the estimated frame transform comprises:

determining an image-to-tool frame relationship between the image frame and the tool frame based on the one or more images; and

computing the estimated frame transform based on the image-to-tool frame relationship.

5. The system of claim 4 , wherein determining the image-to-tool frame relationship further comprises:

identifying pose-indicating points of the tool in the one or more images; and

determining the image-to-tool frame relationship using the pose-indicating points.

6. The system of claim 4 , wherein determining the image-to-tool frame relationship comprises:

determining point pairs derived from a single time instant.

7. The system of claim 4 , wherein determining the image-to-tool frame relationship comprises:

determining point pairs derived from a number of time instants.

8. The system of claim 1 , wherein the method further comprises:

specifying a sequence of frames from the image frame to the tool frame, the sequence of frames including the image frame, the tool frame, and one or more additional frames between the image frame and the tool frame; and

specifying a plurality of frame transforms, each frame transform of the plurality of frame transforms corresponding to a pair of frames of the sequence of frames, the plurality of frame transforms including the unknown frame transform.

9. The system of claim 8 ,

wherein at least one parameter of the unknown frame transform is known, and

wherein determining the estimated frame transform comprises: estimating an unknown parameter of the unknown frame transform.

10. The system of claim 8 ,

wherein the plurality of frame transforms further comprises one or more known frame transforms, and

wherein determining the output movement for the tool comprises: using the one or more known frame transforms in addition to the estimated frame transform.

11. The system of claim 8 , further comprising:

a display viewable by an operator and associated with an eye frame;

wherein the control device has a control device frame; and

wherein determining the output movement for the tool in response to the input at the control device comprises:

determining a master-to-tool transform by: using the estimated frame transform and one or more known frame transforms of the plurality of frame transforms, and aligning a master-to-eye transform with a tool-to-image transform, wherein the master-to-eye transform transforms points in the control device frame to corresponding points in the eye frame, and wherein the tool-to-image transform transforming points in the tool frame to corresponding points in the image frame, and

using the master-to-tool transform to determine the output movement in response to the input.

12. The system of claim 8 , wherein the sequence of frames comprises:

a sequence of frames from the image frame to a fixed frame; and

a second sequence of frames from the fixed frame to the tool frame.

13. The system of claim 8 ,

wherein the plurality of frame transforms comprises a second unknown frame transform; and

wherein the method further comprises: determining a second estimated frame transform for the second unknown frame transform, using the estimated frame transform and one or more known frame transforms of the plurality of frame transforms; and

wherein determining the output movement for the tool in response to the input at the control device comprises: using the second estimated frame transform in addition to the estimated frame transform.

14. A method for operating a system comprising a control device and a manipulator configured to support a tool having a tool frame, the method comprising:

receiving one or more images captured by an image-capturing system, the image-capturing system having an image frame, wherein the tool is visible in the one or more images;

determining an estimated frame transform based on information in the one or more images, wherein the estimated frame transform is used in defining an unknown frame transform between the image frame and the tool frame;

determining, in response to an input at the control device, an output movement for the tool based on the estimated frame transform; and

causing movement of the tool based on the output movement.

15. The method of claim 14 , wherein determining the estimated frame transform based on the one or more images comprises:

determining whether to perform a vision measurement on an image of the one or more images; and

in response to determining to perform the vision measurement, using the image of the one or more images in determining the estimated frame transform.

16. The method of claim 14 , wherein determining the output movement for the tool comprises:

temporally smoothing the estimated frame transform.

17. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions that, when executed by one or more processors associated with a system comprising a control device and a manipulator configured to support a tool having a tool frame, causes the one or more processors to perform a method comprising:

receiving one or more images captured by an image-capturing system, the image-capturing system having an image frame, wherein the tool is visible in the one or more images;

determining an estimated frame transform based on information in the one or more images, wherein the estimated frame transform is used in defining an unknown frame transform between the image frame and the tool frame;

determining, in response to an input at the control device, an output movement for the tool based on the estimated frame transform; and

causing movement of the tool based on the output movement.

18. The non-transitory machine-readable medium of claim 17 , wherein determining the estimated frame transform based on the one or more images comprises:

determining whether to perform a vision measurement on an image of the one or more images; and

in response to determining to perform the vision measurement, using the image of the one or more images in determining the estimated frame transform.

19. The non-transitory machine-readable medium of claim 18 , wherein determining the output movement for the tool comprises:

temporally smoothing the estimated frame transform.