Methods and systems for calibrating instruments within an imaging system, such as a surgical imaging system
Methods and systems for calibrating an instrument, such as a surgical instrument, within an imaging system are disclosed herein. In some embodiments, a method includes capturing images of the instrument with a plurality of cameras of the imaging system and identifying common features of the instrument in the captured images. The method further includes generating a three-dimensional (3D) representation of the instrument based on the common features and determining a reference frame of the instrument based on the generated 3D representation of the instrument. A first transform is determined between the reference frame of the instrument and a reference frame of the cameras. Then, a second transform between the reference frame of the instrument and a reference frame of the tracking structure can be determined based on the first transform.
1. A method of tracking an instrument through a scene, the method comprising:
capturing images of the instrument with a plurality of cameras of an imaging system;
identifying common features of the instrument in the captured images;
generating a three-dimensional (3D) representation of the instrument based on the common features;
determining a reference frame of the instrument based on the generated 3D representation of the instrument;
determining a first transform between the reference frame of the instrument and a reference frame of the cameras;
determining a second transform between the reference frame of the cameras and a reference frame of a plurality of trackers of the imaging system;
determining a third transform between the reference frame of the trackers and a reference frame of a tracking structure attached to the instrument;
determining a fourth transform between the reference frame of the instrument and the reference frame of the tracking structure based on the first transform, the second transform, and the third transform;
tracking a position and orientation of the tracking structure through the scene with the trackers; and
applying the fourth transform to the position and orientation of the tracking structure to track a position and orientation of the instrument.
2. The method of claim 1 wherein the method further comprises:
capturing images of the scene with the cameras;
generating an output image of the scene based on the captured images of the scene; and
displaying a representation of the instrument within the output image based on the second transform, the third transform, and the fourth transform.
3. The method of claim 1 wherein the cameras and the trackers are fixed in position and orientation relative to one another.
4. The method of claim 1 wherein identifying the common features includes automatically identifying the commons features of the instrument in the captured images using a machine learning algorithm.
5. The method of claim 1 wherein determining the fourth transform includes computing a product of the first transform, the second transform, and the fourth-third transform.
6. The method of claim 1 wherein the tracking structure is an optical tracking structure.
7. The method of claim 1 where the instrument is a surgical instrument.
8. The method of claim 1 wherein capturing the images of the instrument includes capturing the images of the instrument against a chromakey background.
9. The method of claim 1 wherein capturing the images of the instrument includes capturing the images of the instrument against a light-emitting source background.
10. The method of claim 1 wherein identifying the common features of the instrument includes receiving a user input manually identifying the commons features of the instrument in the captured images.
11. The method of claim 1 wherein identifying the common features of the instrument includes automatically identifying the commons features of the instrument in the captured images using a feature extraction and segmentation algorithm.
12. The method of claim 1 wherein identifying the common features includes automatically identifying the commons features of the instrument in the captured images using a machine learning algorithm.
13. The method of claim 1 wherein generating the 3D representation of the instrument includes triangulating the common features in 3D space.
14. An imaging system, comprising:
a plurality of cameras configured to capture images of an instrument having a tracking structure attached thereto;
a plurality of trackers configured to capture positional data of the tracking structure;
a processing device communicatively coupled to the cameras and the trackers; and
a non-transitory computer readable storage medium storing instructions that, when executed by the processing device, cause the processing device to identify common features of the instrument in the captured images;
generate a three-dimensional (3D) representation of the instrument based on the common features;
determine a reference frame of the instrument based on the generated 3D representation of the instrument;
determine a first transform between the reference frame of the instrument and a reference frame of the cameras;
determine a second transform between the reference frame of the cameras and a reference frame of the trackers;
determine a third transform between the reference frame of the trackers and a reference frame of the tracking structure;
determine a fourth transform between the reference frame of the instrument and the reference frame of the tracking structure based on the first transform, the second transform, and the third transform;
track a position and orientation of the tracking structure through the scene with the trackers; and
apply the fourth transform to the position and orientation of the tracking structure to track a position and orientation of the instrument.
15. The imaging system of claim 14 wherein the instructions, when executed by the processing device, further cause the processing device to determine the fourth transform by computing a product of the first transform, the second transform, and the third transform.
16. The imaging system of claim 14 wherein the cameras and the trackers are fixedly mounted to a common frame.
17. The imaging system of claim 14 wherein the cameras and the trackers are fixed in position and orientation relative to one another.
18. The imaging system of claim 14 wherein the instructions, when executed by the processing device, further cause the processing device to identify the common features of the instrument by receiving a user input manually identifying the commons features of the instrument in the captured images.
19. The imaging system of claim 14 wherein the instructions, when executed by the processing device, further cause the processing device to identify the commons features of the instrument in the captured images using a feature extraction and segmentation algorithm.
20. The imaging system of claim 14 wherein the instructions, when executed by the processing device, further cause the processing device to identify the common features by automatically identifying the commons features of the instrument in the captured images using a machine learning algorithm.