System and method for image based registration and calibration
Systems and methods for defining a relationship between a surgical object and a tracker coupled to the surgical object. A first optical sensor senses light in an infrared spectrum or a near-infrared spectrum. A second optical sensor senses light in a visible light spectrum. Controller(s) acquire one or more first images of the tracker from the first optical sensor and acquire one or more second images of the surgical object from the second optical sensor. The controller(s) recognize a pose of the tracker based on the one or more first image and recognize a geometry of the surgical object based on the one or more second images. The controller(s) correlate the recognized pose of the tracker and the recognized geometry of the surgical object to define a relationship between the tracker and the surgical object.
1 . A tracking system comprising:
a tracker coupled to a surgical object;
a first optical sensor configured to sense light in an infrared spectrum or a near-infrared spectrum;
a second optical sensor configured to sense light in a visible light spectrum; and
one or more controllers coupled to the first and second optical sensors and being configured to:
acquire one or more first images of the tracker captured at a first exposure from the first optical sensor;
recognize a pose of the tracker based on the one or more first images;
acquire one or more second images of the surgical object captured at a second exposure from the first optical sensor, wherein the first exposure is at least one of shorter in duration and lower in illumination than the second exposure;
recognize a geometry of the surgical object based on the one or more second images; and
correlate the recognized pose of the tracker and the recognized geometry of the surgical object to define a relationship between the tracker and the surgical object.
2 . The tracking system of claim 1 , wherein the tracker has a marker arrangement, and wherein the one or more controllers are configured to:
recognize the pose of the tracker through recognition of a pose of the marker arrangement based on the one or more first images;
associate virtual tracker data with the recognized pose of the marker arrangement;
associate virtual object data with the recognized geometry of the surgical object; and
define the relationship between the tracker and the surgical object through correlation of the virtual tracker data and the virtual object data.
3 . The tracking system of claim 2 , wherein the one or more controllers are configured to retrieve the virtual tracker data from a database comprising a plurality of predefined marker arrangements.
4 . The tracking system of claim 2 , wherein the one or more controllers are configured to generate the virtual tracker data based on the one or more first images.
5 . The tracking system of claim 2 , wherein the one or more controllers are configured to retrieve the virtual object data from a database comprising a plurality of surgical object models.
6 . The tracking system of claim 2 , wherein the one or more controllers are configured to generate the virtual object data based on the one or more second images.
7 . The tracking system of claim 1 , wherein:
the tracker is an active tracker comprising markers configured to actively generate and emit light in the infrared spectrum or the near-infrared spectrum; and
the first optical sensor is configured to sense the light actively generated and emitted by the active tracker.
8 . The tracking system of claim 1 , wherein:
the tracker is a passive tracker comprising markers configured to passively reflect light in the infrared spectrum or the near-infrared spectrum; and
the first optical sensor is configured to sense the light passively reflected by the passive tracker.
9 . The tracking system of claim 8 , further comprising an infrared or near-infrared light source for illuminating the markers of the passive tracker, wherein the one or more controllers are configured to cause the light source to illuminate at a first illumination level to acquire the one of more first images at the first exposure and cause the light source to illuminate a second illumination level to acquire the one or more second images at the second exposure, the first illumination level being less than the second illumination level.
10 . The tracking system of claim 1 , wherein the surgical object comprises an anatomy of a patient, wherein the one or more controllers correlate the recognized pose of the tracker and the recognized geometry of the anatomy to define a registration between the tracker and the anatomy.
11 . The tracking system of claim 1 , wherein the surgical object comprises a surgical tool, wherein the one or more controllers correlate the recognized pose of the tracker and the recognized geometry of the surgical tool to calibrate the surgical tool based on the defined relationship.
12 . The tracking system of claim 1 , wherein the surgical object comprises a robotic manipulator.
13 . The tracking system of claim 1 , wherein the first optical sensor is further defined as two optical sensors each configured to sense light in the infrared spectrum or the near-infrared spectrum, wherein the two optical sensors are rigidly mounted to a common support structure and are arranged stereoscopically and separated from one another by at least six inches.
14 . The tracking system of claim 1 , wherein the second optical sensor is a video camera.
15 . The tracking system of claim 1 , wherein the first optical sensor and the second optical sensor are commonly housed in a camera unit.
16 . The tracking system of claim 1 , further comprising a display device coupled to the one or more controllers, and wherein the one or more controllers are configured to present on the display device a representation of the tracker and the surgical object in accordance with the defined relationship.
17 . A method of operating a tracking system, the tracking system including a tracker coupled to a surgical object, a first optical sensor configured to sense light in an infrared spectrum or a near-infrared spectrum, a second optical sensor configured to sense light in a visible light spectrum, and one or more controllers coupled to the first and second optical sensors, the method comprising the one or more controllers performing the following steps:
acquiring one or more first images of the tracker captured at a first exposure from the first optical sensor;
recognizing a pose of the tracker based on the one or more first images;
acquiring one or more second images of the surgical object captured at a second exposure from the first optical sensor, wherein the first exposure is at least one of shorter in duration and lower in illumination than the second exposure;
recognizing a geometry of the surgical object based on the one or more second images; and
correlating the recognized pose of the tracker and the recognized geometry of the surgical object for defining a relationship between the tracker and the surgical object.
18 . The method claim 17 , wherein the tracker has a marker arrangement, and comprising the one or more controllers:
recognizing the pose of the tracker by recognizing a pose of the marker arrangement based on the one or more first images;
associating virtual tracker data with the recognized pose of the marker arrangement;
associating virtual object data with the recognized geometry of the surgical object; and
defining the relationship between the tracker and the surgical object by correlating the virtual tracker data and the virtual object data.
19 . The method of claim 18 , comprising the one or more controllers retrieving the virtual tracker data from a database comprising a plurality of predefined marker arrangements.
20 . The method of claim 18 , comprising the one or more controllers generating the virtual tracker data based on the one or more first images.
21 . The method of claim 18 , comprising the one or more controllers retrieving the virtual object data from a database comprising a plurality of surgical object models.
22 . The method of claim 18 , comprising the one or more controllers generating the virtual object data based on the one or more second images.
23 . The method of claim 17 , comprising the one or more controllers presenting, on a display device, a representation of the tracker and the surgical object in accordance with the defined relationship.
24 . A non-transitory computer readable medium configured to define a relationship between a surgical object and a tracker coupled to the surgical object, wherein the non-transitory computer readable medium comprises instructions, which when executed by one or more processors, are configured to:
acquire one or more first images of the tracker captured at a first exposure in an infrared spectrum or a near-infrared spectrum;
analyze the one or more first images to estimate a pose of the tracker;
acquire one or more second images of the surgical object captured as a second exposure in the infrared spectrum or near-infrared spectrum, wherein the first exposure is at least one of shorter in duration and lower in illumination than the second exposure;
analyze the one or more second images to estimate a geometry of the surgical object; and
correlate the estimated pose of the tracker and the estimated geometry of the surgical object to define the relationship between the tracker and the surgical object.
25 . A computer-implemented method for defining a relationship between a surgical object and a tracker coupled to the surgical object, the computer-implemented method comprising:
acquiring one or more first images of the tracker captured at a first exposure in an infrared spectrum or a near-infrared spectrum;
analyzing the one or more first images for estimating a pose of the tracker;
acquiring one or more second images of the surgical object captured at a second exposure in the infrared spectrum of near-infrared spectrum, wherein the first exposure is at least one of shorter in duration and lower in illumination than the second exposure;
analyzing the one or more second images for estimating a geometry of the surgical object; and
correlating the estimated pose of the tracker and the estimated geometry of the surgical object for defining the relationship between the tracker and the surgical object.