IP Library Granted Patent US 11,707,330
Granted Patent B2
US 11,707,330 · App. 16/674,447 · Granted Jul 25, 2023

Systems and methods for surgical navigation

Inventors: Jeremy Weinstein (Santa Clara, CA); Andrei Danilchenko (Miami, FL); Jose Luis Moctezuma de la Barrera (Freiburg, DE)
Assignee: MAKO Surgical Corp.
A61B34/20A61B90/361A61F2/461A61B34/30A61B2034/105A61B2034/107A61B2034/2048A61B2034/2055A61B2034/2057A61B2034/2059A61B2034/2065A61B2034/2068A61B2034/2072A61B2034/252A61B2090/372A61B2090/373A61B2090/395A61B2090/3983A61B2090/502A61F2002/4633
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,707,330
App. No.
16/674,447
Granted
Jul 25, 2023
Kind
B2
Abstract

Disclosed are systems, methods, and techniques for registering a HMD coordinate system of a head-mounted display (HMD) and a localizer coordinate system of a surgical navigation localizer. A camera of the HMD captures at least one image of a registration device having a registration coordinate system and a plurality of registration markers. The registration markers are analyzed in the at least one image to determine a pose of the HMD coordinate system relative to the registration coordinate system. One or more position sensors comprised in the localizer detect a plurality of tracking markers comprised in the registration device to determine a pose of the registration coordinate system relative to the localizer coordinate system. The HMD coordinate system and the localizer coordinate system are registered using the registration device, wherein positions of the registration markers are known with respect to positions of the tracking markers in the registration coordinate system.

Claims (37)

1. A registration and tracking device configured for enabling registration of a head-mounted display (HMD) and tracking of a surgical object by a surgical navigation localizer, the registration and tracking device comprising:

a support frame configured to be coupled to the surgical object;

a plurality of registration markers coupled to the support frame and being arranged in a first pattern to define a registration coordinate system, the plurality of registration markers configured to be analyzed in an image captured by a camera of the HMD, operable in a HMD coordinate system, to determine a pose of the HMD coordinate system relative to the registration coordinate system; and

a plurality of tracking markers coupled to the support frame and being arranged in a second pattern to define a tracker coordinate system, wherein the registration coordinate system and the tracker coordinate system have a known relationship, the plurality of tracking markers configured to be detected by one or more position sensors of the surgical navigation localizer having a localizer coordinate system, to determine a pose of the registration coordinate system relative to the localizer coordinate system and to track the surgical object in the localizer coordinate system.

2. The registration and tracking device of claim 1 , wherein the plurality of registration markers comprises three or more registration markers.

3. The registration and tracking device of claim 1 , wherein the plurality of tracking markers comprises three or more tracking markers.

4. The registration and tracking device of claim 1 , wherein the plurality of tracking markers is for being sensed by an electromagnetic tracking modality.

5. The registration and tracking device of claim 1 , wherein the support frame is configured to be coupled to the surgical object being further defined as: a patient anatomy, a surgical tool, or a robotic manipulator.

6. The registration and tracking device of claim 1 , wherein the plurality of tracking markers is for being sensed by a camera of the surgical navigation localizer.

7. The registration and tracking device of claim 6 , wherein the tracking markers are passive tracking markers reflecting light or active tracking markers transmitting light.

8. The registration and tracking device of claim 1 , wherein the plurality of registration markers comprises markers having a known pattern for being recognized in the image captured by the camera of the HMD.

9. The registration and tracking device of claim 1 , wherein the plurality of registration markers comprises markers having a unique pattern for being recognized in the image captured by the camera of the HMD.

10. The registration and tracking device of claim 1 , wherein the plurality of registration markers comprises markers having a known, unique pattern for being recognized in the image captured by the camera of the HMD.

11. The registration and tracking device of claim 1 , wherein the plurality of registration markers comprises printed markers.

12. The registration and tracking device of claim 11 , wherein each of the printed markers have unique patterns.

13. The registration and tracking device of claim 1 , wherein the plurality of registration markers is for being analyzed in the image captured by the camera of the HMD and for being detected by a depth sensor of the HMD, to determine the pose of the HMD coordinate system relative to the registration coordinate system.

14. A surgical navigation system comprising:

a head-mounted display (HMD) operable in a HMD coordinate system, the HMD comprising a camera;

a surgical navigation localizer having a localizer coordinate system, and comprising one or more position sensors;

at least one controller coupled to the HMD and the surgical navigation localizer; and

a registration and tracking device comprising: a support frame configured to be coupled to a surgical object, a plurality of registration markers coupled to the support frame and being arranged in a first pattern to define a registration coordinate system, and a plurality of tracking markers coupled to the support frame and being arranged in a second pattern to define a tracker coordinate system, wherein the registration coordinate system and the tracker coordinate system have a known transform;

the HMD being configured to capture an image of the plurality of registration markers with the camera and the at least one controller being configured to analyze the plurality of registration markers in the captured image and determine a first transform between the HMD coordinate system and the registration coordinate system;

the surgical navigation localizer being configured to detect the plurality of tracking markers with the one or more position sensors and the at least one controller being configured to determine a second transform between the tracker coordinate system and the localizer coordinate system; and

the at least one controller to register the HMD coordinate system and the localizer coordinate system by being configured to combine the first transform, the second transform and the known transform; and

wherein the surgical navigation localizer is further configured to detect the plurality of tracking markers with the one or more position sensors to track the surgical object in the localizer coordinate system.

15. The surgical navigation system of claim 14 , wherein the HMD comprises a tracker that is detectable by the one or more position sensors of the surgical navigation localizer.

16. The surgical navigation system of claim 14 , wherein the HMD coordinate system comprises a local coordinate system of the HMD.

17. A method of operating a surgical navigation system, the surgical navigation system comprising a head-mounted display (HMD) operable in a HMD coordinate system, the HMD comprising a camera, a surgical navigation localizer having a localizer coordinate system and comprising one or more position sensors, at least one controller coupled to the HMD and the surgical navigation localizer; and a registration and tracking device comprising a support frame configured to be coupled to a surgical object, a plurality of registration markers coupled to the support frame and being arranged in a first pattern to define a registration coordinate system, and a plurality of tracking markers coupled to the support frame and being arranged in a second pattern to define a tracker coordinate system, wherein the registration coordinate system and the tracker coordinate system have a known transform, the method comprising:

capturing, with the camera of the HMD, an image of the plurality of registration markers;

analyzing, with the at least one controller, the plurality of registration markers in the captured image for determining a first transform between the HMD coordinate system and the registration coordinate system;

detecting, with the one or more position sensors of the surgical navigation localizer, the plurality of tracking markers;

determining, with the at least one controller, a second transform between the tracker coordinate system and the localizer coordinate system;

registering, with the at least one controller, the HMD coordinate system and the localizer coordinate system by combining the first transform, the second transform and the known transform; and

detecting, with the one or more position sensors of the surgical navigation localizer, the plurality of tracking markers for tracking the surgical object in the localizer coordinate system.

18. The method of claim 17 , wherein the step of registering the HMD coordinate system and the localizer coordinate system is performed in response to a user directing the HMD toward the registration markers so that the registration markers are within the image captured by the camera and in response to the one or more position sensors detecting the tracking markers.

19. The method of claim 17 , wherein the step of capturing the image and the step of detecting the plurality of tracking markers are performed at different points in time.

20. The method of claim 17 , comprising the at least one controller further determining the first transform between the HMD coordinate system relative to the registration coordinate system based on data of a depth sensor comprised in the HMD.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 15, 2020
From: WEINSTEIN, JEREMY; DANILCHENKO, ANDREI
To: MAKO SURGICAL CORP.
Reel/Frame 052400/0028 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 15, 2020
From: MOCTEZUMA DE LA BARRERA, JOSE LUIS
To: STRYKER LEIBINGER GMBH & CO. KG
Reel/Frame 052400/0057 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 15, 2020
From: STRYKER LEIBINGER GMBH & CO. KG
To: MAKO SURGICAL CORP.
Reel/Frame 052400/0166 →
Continuity (3)
Continuation 15860057 · Jan 2, 2018
Provisional Application 62441713 · Jan 3, 2017
Related Publication 20200078100A1 · Mar 12, 2020
Cited By (2)
US 12,383,347 US 12,629,307