IP Library Granted Patent US 11,806,090
Granted Patent B2
US 11,806,090 · App. 17/679,241 · Granted Nov 7, 2023

System and method for image based registration and calibration

Inventors: Jean Stawiaski (Kirchzarten, DE); Fadi Ghanam (Schallstadt, DE); David Hofmann (Freiburg, DE)
Assignee: MAKO Surgical Corp.
A61B34/20A61B17/16A61B34/10A61B34/30A61B34/70G06T7/00G06T7/74A61B2017/1602A61B2034/101A61B2034/107A61B2034/2051A61B2034/2063A61B2034/2065A61B2034/2068G06V2201/03
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,806,090
App. No.
17/679,241
Granted
Nov 7, 2023
Kind
B2
Abstract

Systems and methods for calibration of a surgical tool having a tracker. At least one camera captures images of the tracker at a first exposure and captures images of the surgical tool at a second exposure with a different exposure time and/or illumination than the first exposure. Controller(s) recognize a pose of the tracker based on the first exposure images and recognize a geometry of the surgical tool based on the second exposure images. The controller(s) correlate the recognized pose of the tracker and the recognized geometry of the surgical tool to define a relationship between the tracker and the surgical tool and calibrate the surgical tool based on the defined relationship.

Claims (80)

1. A method of calibrating a surgical tool using a system comprising at least one infrared camera and one or more controllers, and wherein a tracker is coupled to the surgical tool for movement therewith, the method comprising the one or more controllers performing the steps of:

acquiring one or more first exposure infrared images of the tracker captured at a first exposure with at least one infrared camera;

recognizing a pose of the tracker based on the one or more first exposure infrared images;

acquiring one or more second exposure infrared images of the surgical tool captured at a second exposure with the at least one infrared camera, wherein the second exposure is different than the first exposure with respect to one or both of: exposure time and level of infrared illumination;

recognizing a geometry of the surgical tool based on the one or more second exposure infrared images;

correlating the recognized pose of the tracker and the recognized geometry of the surgical tool to define a relationship between the tracker and the surgical tool; and

calibrating the surgical tool based on the defined relationship.

2. The method of claim 1 , wherein the tracker has a marker arrangement, and comprising the one or more controllers performing the steps of:

recognizing a pose of the marker arrangement based on the one or more first exposure infrared images;

associating virtual tracker data with the recognized pose of the marker arrangement;

associating virtual tool data with the recognized geometry of the surgical tool; and

correlating the virtual tracker data and the virtual tool data to define the relationship between the tracker and the surgical tool.

3. The method of claim 2 , comprising the one or more controllers performing the step of:

retrieving the virtual tracker data from a database comprising a plurality of predefined marker arrangements; or

generating the virtual tracker data based on the one or more first exposure infrared images.

4. The method of claim 2 , comprising the one or more controllers performing the step of:

retrieving the virtual tool data from a database comprising a plurality of surgical tool models; or

generating the virtual tool data based on the one or more second exposure infrared images.

5. The method of claim 1 , wherein the surgical tool comprises a tool axis, and comprising the one or more controllers performing the steps of:

recognizing a geometry of the tool axis based on the one or more second exposure infrared images;

correlating the recognized pose of the tracker and the recognized geometry of the tool axis to define a relationship between the tracker and the tool axis; and

calibrating the surgical tool based on the defined relationship between the tracker and the tool axis.

6. The method of claim 1 , wherein the surgical tool comprises a tool tip, and comprising the one or more controllers performing the steps of:

recognizing a geometry of the tool tip based on the one or more second exposure infrared images;

correlating the recognized pose of the tracker and the recognized geometry of the tool tip to define a relationship between the tracker and the tool tip; and

calibrating the surgical tool based on the defined relationship between the tracker and the tool tip.

7. The method of claim 1 , comprising the one or more controllers recognizing the geometry of the surgical tool by utilizing one or both of:

a detection algorithm for recognizing ridges and edges of the surgical tool; and

a Hough transform for recognizing a shape of the surgical tool.

8. The method of claim 1 , comprising the one or more controllers performing the steps of:

acquiring one or more second exposure infrared images of the tracker captured at the second exposure with the at least one infrared camera;

recognizing a geometry of the tracker based on the one or more second exposure infrared images of the tracker; and

correlating the recognized pose and geometry of the tracker and the recognized geometry of the surgical tool to define the relationship between the tracker and the surgical tool.

9. The method of claim 1 , comprising the one or more controllers performing the steps of:

recognizing a pose of the surgical tool based on the one or more second exposure infrared images; and

correlating the recognized pose of the tracker and the recognized geometry and pose of the surgical tool to define the relationship between the tracker and the surgical tool.

10. A system for calibrating a surgical tool, wherein a tracker is coupled to the surgical tool for movement therewith, the system comprising:

at least one infrared camera being configured to:

capture one or more first exposure infrared images of the tracker at a first exposure; and

capture one or more second exposure infrared images of the surgical tool at a second exposure, wherein the second exposure is different than the first exposure with respect to one or both of: exposure time and level of infrared illumination; and

one or more controllers coupled to the at least one infrared camera and being configured to:

recognize a pose of the tracker based on the one or more first exposure infrared images;

recognize a geometry of the surgical tool based on the one or more second infrared exposure images;

correlate the recognized pose of the tracker and the recognized geometry of the surgical tool to define a relationship between the tracker and the surgical tool; and

calibrate the surgical tool based on the defined relationship.

11. The system of claim 10 , wherein the tracker has a marker arrangement, and wherein the one or more controllers are configured to:

recognize a pose of the marker arrangement based on the one or more first exposure infrared images;

associate virtual tracker data with the recognized pose of the marker arrangement;

associate virtual tool data with the recognized geometry of the surgical tool; and

correlate the virtual tracker data and the virtual tool data to define the relationship between the tracker and the surgical tool.

12. The system of claim 11 , wherein the one or more controllers are configured to:

retrieve the virtual tracker data from a database comprising a plurality of predefined marker arrangements; or

generate the virtual tracker data based on the one or more first exposure infrared images.

13. The system of claim 11 , wherein the one or more controllers are configured to:

retrieve the virtual tool data from a database comprising a plurality of surgical tool models; or

generate the virtual tool data based on the one or more second exposure infrared images.

14. The system of claim 10 , wherein the surgical tool comprises a tool axis, and wherein the one or more controllers are configured to:

recognize a geometry of the tool axis based on the one or more second exposure infrared images;

correlate the recognized pose of the tracker and the recognized geometry of the tool axis to define a relationship between the tracker and the tool axis; and

calibrate the surgical tool based on the defined relationship between the tracker and the tool axis.

15. The system of claim 10 , wherein the surgical tool comprises a tool tip, and wherein the one or more controllers are configured to:

recognize a geometry of the tool tip based on the one or more second exposure infrared images;

correlate the recognized pose of the tracker and the recognized geometry of the tool tip to define a relationship between the tracker and the tool tip; and

calibrate the surgical tool based on the defined relationship between the tracker and the tool tip.

16. The system of claim 10 , wherein the second exposure has an exposure time that is longer in duration than an exposure time of the first exposure.

17. The system of claim 10 , wherein the second exposure is greater in infrared illumination than the first exposure.

18. The system of claim 10 , wherein the at least one infrared camera comprises a first infrared camera with a single optical sensor, and wherein the first infrared camera is configured to:

capture the one or more first exposure infrared images at the first exposure; and

capture the one or more second exposure infrared images at the second exposure.

19. The system of claim 10 , wherein the at least one infrared camera comprises first and second infrared cameras rigidly mounted to a common support structure and separated by a separation distance, wherein the separation distance is selected from among: greater than six inches, greater than eight inches, greater than twelve inches, or greater than twenty-four inches.

20. A surgical system comprising:

a surgical tool;

a tracker coupled to the surgical tool;

at least one infrared camera being configured to:

capture one or more first exposure infrared images of the tracker at a first exposure; and

capture one or more second exposure infrared images of the surgical tool at a second exposure, wherein the second exposure is different than the first exposure with respect to one or both of: exposure time and level of infrared illumination; and

one or more controllers coupled to the at least one infrared camera and being configured to:

recognize a pose of the tracker based on the one or more first exposure infrared images;

recognize a geometry of the surgical tool based on the one or more second exposure infrared images; and

correlate the recognized pose of the tracker and the recognized geometry of the surgical tool to define a relationship between the tracker and the surgical tool.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2022
From: STRYKER LEIBINGER GMBH & CO. KG
To: MAKO SURGICAL CORP.
Reel/Frame 059149/0059 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2022
From: STAWIASKI, JEAN; GHANAM, FADI; HOFMANN, DAVID
To: STRYKER LEIBINGER GMBH & CO. KG
Reel/Frame 059148/0969 →
Continuity (3)
Continuation 16441624 · Jun 14, 2019
Provisional Application 62698502 · Jul 16, 2018
Related Publication 20220175467A1 · Jun 9, 2022
Cited By (1)
US 12,642,597