Devices, systems and methods for natural feature tracking of surgical tools and other objects
Systems, methods and devices for use in tracking are described, using optical modalities to detect spatial attributes or natural features of objects, such as, tools and patient anatomy. Spatial attributes or natural features may be known or may be detected by the tracking system. The system, methods and devices can further be used to verify a calibration of a tool either by a computing unit or by a user. Further, the disclosure relates to detection of spatial attributes, including depth information, of the anatomy for purposes of registration or to create a 3D surface profile of the anatomy.
1 . A system to determine object poses during a surgical navigation procedure, the system comprising:
a sensor unit comprising an optical sensor configured to generate images of objects within a field of view of the optical sensor; and
a computer processing unit, in communication with the sensor unit and memory storing instructions that, when executed by the processing unit, cause the system to perform a method comprising:
storing first natural feature information of a first object, the natural feature information defining information to identify the first object in the images using natural visual features of the first object;
receiving from the sensor unit an image of the first object;
extracting second natural feature information from natural visual features of the first object in the image; and
calculating the pose of the first object relative to the sensor unit using the first natural feature information and the second natural feature information to facilitate performance of the surgical navigation procedure;
wherein each of the first natural feature information and second natural feature information is defined from one or more of the natural visual features comprising inherent spatial features comprising any of a shape feature, a pattern feature, a configuration feature or an ornamentation feature; and
wherein the one or more of the natural visual features relate to portions of the first object without including any markers or identifiers applied to the first object for purposes of tracking the first object.
2 . The system of claim 1 , wherein the sensor unit is configured for attachment to a portion of a patient's anatomy.
3 . The system of claim 2 , wherein the portion is a pelvic bone.
4 . The system of claim 1 , wherein the first object is an acetabular impactor.
5 . The system of claim 1 , wherein the instructions cause the system to simultaneously localize targets and natural visual features from the image received from the sensor unit.
6 . The system of claim 5 , wherein:
the image received includes the first object and a second object, the second object having a target coupled thereto; and
the instructions cause the system to:
localize the first object via the first natural feature information and the second natural feature information; and
localize the second object from target information obtained from the image and stored target information for the target.
7 . The system of claim 6 , wherein the first object comprises a first instrument and the second object a second instrument and wherein the instructions cause the system to determine poses of each of the first instrument and second instrument.
8 . The system of claim 6 , wherein:
the first object is an acetabular impactor;
the second object is a patient leg; and
the instructions cause the system to:
localize the acetabular impactor relative to a patient pelvis; and
localize a position of the patient leg relative to the patient pelvis using a target attached to the leg.
9 . The system of claim 1 , wherein the instructions cause the system to obtain the first natural feature information from a digital file accessible to the processing unit.
10 . The system of claim 9 , wherein the first object is a tool comprising an effector, and the first natural feature information includes a positional relationship of the effector with respect to other parts of the tool.
11 . The system of claim 1 , wherein to calculate the pose the instructions cause the system to perform optimization operations to: match at least some of the first natural feature information and at least some of the second natural feature information; and calculate the pose responsive to the match.
12 . The system of claim 1 , wherein the instructions cause the system to compute the first natural feature information using a feature learning procedure in which sufficient views of the object are presented to the processing unit for learning its natural visual features.
13 . The system of claim 1 , wherein the sensor is configured with a light to illuminate the natural visual features in the field of view.
14 . The system of claim 1 , wherein the first natural feature information is derived from a medical image of the patient comprising a Computed Tomography (CT) scan, a Magnetic Resonance Image (MRI) scan, or another pre-operative scan of the patient.
15 . A system to determine object poses during a surgical navigation procedure, the system comprising:
a sensor unit comprising an optical sensor configured to generate images of objects within a field of view of the optical sensor; and
a computer processing unit, in communication with the sensor unit and memory storing instructions that, when executed by the processing unit, cause the system to perform a method comprising:
storing first natural feature information of a first object, the natural feature information defining information to identify the first object in the images using natural visual features of the first object;
receiving from the sensor unit an image of the first object;
extracting second natural feature information from natural visual features of the first object in the image; and
calculating the pose of the first object relative to the sensor unit using the first natural feature information and the second natural feature information to facilitate performance of the surgical navigation procedure;
wherein the pose is calculated in any number of degrees of freedom between 2 degrees of freedom and 6 degrees of freedom.
16 . A system to determine object poses during a surgical navigation procedure, the system comprising:
a sensor unit comprising an optical sensor configured to generate images of objects within a field of view of the optical sensor; and
a computer processing unit, in communication with the sensor unit and memory storing instructions that, when executed by the processing unit, cause the system to perform a method comprising:
storing first natural feature information of a first object, the natural feature information defining information to identify the first object in the images using natural visual features of the first object;
receiving from the sensor unit an image of the first object;
extracting second natural feature information from natural visual features of the first object in the image; and
calculating the pose of the first object relative to the sensor unit using the first natural feature information and the second natural feature information to facilitate performance of the surgical navigation procedure;
wherein the first natural feature information and the second natural feature information each comprises information defining edges of a cylindrical shaft of the first object.
17 . The system of claim 16 , wherein operations to calculate the pose account for a perspective effect of the natural visual features in the image.
18 . The system of claim 16 , wherein the first natural feature information is derived from a medical image of the patient comprising a Computed Tomography (CT) scan, a Magnetic Resonance Image (MRI) scan, or another pre-operative scan of the patient.
19 . A system to determine object poses during a surgical navigation procedure, the system comprising:
a sensor unit comprising an optical sensor configured to generate images of objects within a field of view of the optical sensor; and
a computer processing unit, in communication with the sensor unit and memory storing instructions that, when executed by the processing unit, cause the system to perform a method comprising:
storing first natural feature information of a first object, the natural feature information defining information to identify the first object in the images using natural visual features of the first object;
receiving from the sensor unit an image of the first object;
extracting second natural feature information from natural visual features of the first object in the image; and
calculating the pose of the first object relative to the sensor unit using the first natural feature information and the second natural feature information to facilitate performance of the surgical navigation procedure;
wherein:
the sensor unit is configured to provide first images of the field of view in a first light spectrum and second images of the field of view in a second light spectrum that is different from the first light spectrum;
the image comprises a first image from the first images, the first image including a first object; and
the instructions cause the system to:
track the first object using the natural visual features of the first object;
receive a second image comprising one of the second images, the second image including a target coupled to a second object; and
track the second object using the target;
wherein the first object and second object are tracked relative to a same coordinate reference of the sensor unit.
20 . The system of claim 19 , wherein:
the first object is an acetabular impactor and the first spectrum is a visible light spectrum; the second object is a patient leg and the second spectrum is an infra-red spectrum; and the instructions cause the system to:
localize the acetabular impactor relative to a patient pelvis; and
localize a position of the patient leg relative to the patient pelvis using a target attached to the leg.