Configuration marker design and detection for instrument tracking
The present disclosure relates to systems, methods, and tools for tool tracking using image-derived data from one or more tool-located references features. In some embodiments, a medical system includes a tool having a distal end that is insertable into a patient body, an image capture device insertable into the patient body so that the image capture device captures an image of at least a portion of a two-dimensional marker at least partially surrounding a portion of the tool, and a processor coupled to the image capture device and configured to determine a pose of the tool by processing the image.
1. A medical system comprising:
a tool having a distal end that is insertable into a patient body, wherein the distal end includes a two-dimensional marker at least partially surrounding a portion of the tool, and wherein the two-dimensional marker includes at least four non-collinear features;
an image capture device insertable into the patient body so that the image capture device captures an image of at least a portion of the two-dimensional marker, wherein the image of the at least a portion of the two-dimensional marker includes at least three non-collinear features of the at least four non-collinear features, and wherein the image capture device is one of a stereoscopic image capture device or a monoscopic image capture device; and
a processor coupled to the image capture device and configured to determine a pose of the tool by processing the image.
2. The medical system of claim 1 , wherein the two-dimensional marker completely encircles the portion of the tool.
3. The medical system of claim 1 , wherein the two-dimensional marker includes more than one instance of a repeating pattern.
4. The medical system of claim 3 , wherein the instances of the repeating pattern are disposed at 120 degree intervals around the portion of the tool.
5. The medical system of claim 4 , wherein the instances of the repeating pattern are identical.
6. The medical system of claim 1 , wherein the two-dimensional marker includes features that are aligned with a longitudinal axis of the portion of the tool.
7. The medical system of claim 1 , wherein the two-dimensional marker includes localization features that allow the portion of the tool to be localized within a work site and identification features that allow the tool to be identified from among a plurality of tools.
8. The medical system of claim 1 , wherein the two-dimensional marker includes a pattern having at least one circular feature and at least one bar feature.
9. A robotic tool comprising:
an elongate body having a proximal end and a distal end, the proximal end being coupled to a back end structure and the distal end being insertable into a patient body;
an end effector coupled to the elongate body;
a hinge that couples that end effector to the elongate body; and
a two-dimensional marker at least partially surrounding a portion of the elongate body, wherein the two-dimensional marker includes at least two instances of a repeating pattern, a first instance of the repeating pattern being disposed at a 120 degree interval around the portion of the robotic tool with respect to a second instance of the repeating pattern, and wherein each of the first instance of the repeating pattern and the second instance of the repeating pattern includes at least four non-collinear features.
10. The robotic tool of claim 9 , wherein the two-dimensional marker includes features that are aligned with a longitudinal axis of the portion of the elongate body.
11. The robotic tool of claim 9 , wherein the two-dimensional marker includes localization features that allow the portion of the elongate body to be localized within a work site and identification features that allow the robotic tool to be identified from among a plurality of tools.
12. A robotic tool tracking method comprising:
receiving image data from an image capture device insertable into a cavity, the image data including an image of at least a portion of a two-dimensional marker at least partially surrounding a portion of a tool having a distal end that is insertable into the cavity, wherein the two-dimensional marker includes at least four non-collinear features, and wherein the image of the at least a portion of the two-dimensional marker includes at least three non-collinear features of the at least four non-collinear features, and wherein the image capture device is one of a stereoscopic image capture device or a monoscopic image capture device;
generating an estimated tool state for the tool by using at least one prior tool state from the received image data of the tool for a prior time or joint data from a robotic actuation system effecting movement of the tool; and
determining a pose of the tool by processing information of the stereo image data so as to locate the two-dimensional marker.
13. The method of claim 12 , further comprising illuminating the cavity with a light source insertable into the cavity so as to be oriented to direct light toward the distal end of the tool when the distal end is within the cavity.
14. The method of claim 12 , wherein the cavity is a body cavity in a patient.
15. The method of claim 12 , wherein the two-dimensional marker includes at least one circular feature and at least one bar feature.
16. The method of claim 12 , wherein the image data comprises stereo image data and receiving image data comprises receiving left image data and receiving corresponding right image data, and wherein determining a pose of the tool by processing information of the stereo image data comprises:
extracting primitive features from the left image data and the corresponding right image data, wherein the primitive features include visually salient features within the left image data and the corresponding right image data;
generating a hypothesis of a location of the tool based on localization features included in the extracted primitive features; and
generating a hypothesis of the location and an orientation of the tool based on additional features included in the extracted primitive features.
17. The method of claim 12 , further comprising approximating features extracted from the image data of the two-dimensional marker as a plane.
18. The method of claim 12 , wherein the two-dimensional marker completely encircles the portion of the tool.
19. The method of claim 12 , wherein the two-dimensional marker includes a plurality of instances of a repeating pattern, and wherein the instances are disposed at 120 degree intervals around the portion of the tool with respect to each other.
20. The method of claim 12 , wherein the two-dimensional marker includes features that are aligned with a longitudinal axis of the portion of the tool.