IP Library › Granted Patent US 12,238,433
Granted Patent B2
US 12,238,433 · App. 18/616,266 · Granted Feb 25, 2025

Systems and methods for tracking objects

Inventors: Jochen Breisacher (Teningen, DE); David Hofmann (Freiburg, DE); Emeric Umbdenstock (Freiburg, DE); Jonathan Morgan (Biscayne Park, FL); Paul Hoekstra (Kalamazoo, MI)
Assignee: MAKO Surgical Corp.
H04N25/44A61B34/20H04N13/243H04N23/60H04N23/61H04N25/40H04N25/443A61B2034/2046A61B2034/2055A61B2034/2057A61B2034/2065A61B34/70H04N23/695H04N25/42
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,238,433
App. No.
18/616,266
Granted
Feb 25, 2025
Kind
B2
Abstract

Surgical navigation systems and methods for tracking an object within an operating room involve a camera unit that houses a first stereoscopic sensing system and a second stereoscopic sensing system each comprising sensing elements adapted to sense light. A controller utilizes the first stereoscopic sensing system to sense light from the object and track the object according to a first resolution. The controller utilizes the second stereoscopic sensing system to sense light from the object and track the object according to a second resolution that is lower than the first resolution. The controller obtains data related to the object from the second stereoscopic sensing system and modifies control of the sensing elements of the first stereoscopic sensing system based on the obtained data related to the object.

Claims (55)

1. A surgical navigation system for tracking an object within an operating room, the surgical navigation system comprising:

a camera unit including:

a housing;

a first stereoscopic sensing system coupled to the housing and comprising sensing elements adapted to sense light;

a second stereoscopic sensing system coupled to the housing and comprising sensing elements adapted to sense light; and

a controller in communication with the first and second stereoscopic sensing systems, and wherein the controller is configured to:

utilize the first stereoscopic sensing system to sense light from the object and track the object according to a first resolution;

utilize the second stereoscopic sensing system to sense light from the object and track the object according to a second resolution that is lower than the first resolution;

obtain data related to the object from the second stereoscopic sensing system; and

modify control of the sensing elements of the first stereoscopic sensing system based on the data of the object obtained by the second stereoscopic sensing system.

2. The surgical navigation system of claim 1 , wherein the data related to the object comprises one or more of: a type of the object; a pose of the object; a velocity of the object; an acceleration of the object; and/or an expected movement of the object.

3. The surgical navigation system of claim 1 , wherein the controller is configured to modify control of the sensing elements of the first stereoscopic sensing system to activate a subset of the sensing elements, the subset being less than all of the sensing elements.

4. The surgical navigation system of claim 1 , wherein the controller is configured to modify control of the sensing elements of the first stereoscopic sensing system to prevent processing of a subset of the sensing elements, the subset being less than all of the sensing elements.

5. The surgical navigation system of claim 1 , wherein the controller is configured to modify control of the sensing elements of the first stereoscopic sensing system to adjust a size of a field-of-view viewed by the first stereoscopic sensing system.

6. The surgical navigation system of claim 1 , wherein the controller is configured to modify control of the sensing elements of the first stereoscopic sensing system to define a size and/or position of a region-of-interest viewed by the first stereoscopic sensing system, the region-of-interest being a sub-region within a full field-of-view of the first stereoscopic sensing system.

7. The surgical navigation system of claim 1 , wherein the controller is configured to modify control of the sensing elements of the first stereoscopic sensing system by adjusting a frequency by which the sensing elements are read-out.

8. The surgical navigation system of claim 1 , wherein:

the sensing elements of the first stereoscopic sensing system are adapted to sense light in an infrared spectrum or near-infrared spectrum; and

the sensing elements of the second stereoscopic sensing system are adapted to sense light in a visible light spectrum.

9. The surgical navigation system of claim 1 , wherein:

the first stereoscopic sensing system has a first field-of-view; and

the second stereoscopic sensing system has a second field-of-view that has a size different from a size of the first field-of-view.

10. The surgical navigation system of claim 1 , wherein:

a tracker is coupled to the object and the tracker comprises tracking fiducials; and

the controller is configured to:

utilize the first stereoscopic sensing system to sense light from the tracking fiducials and track the object according to the first resolution; and

utilize the second stereoscopic sensing system to sense light from the object and/or the tracking fiducials and track the object according to the second resolution that is lower than the first resolution.

11. The surgical navigation system of claim 1 , wherein:

the sensing elements of the first stereoscopic sensing system are grouped into sensor units;

the sensing elements of the second stereoscopic sensing system are grouped into sensor units;

the camera unit comprises a rigid support structure disposed within the housing and being configured to rigidly support the sensor units of each of the first and second stereoscopic sensing systems;

the sensor units of the first stereoscopic sensing system are spaced apart from one another on the rigid support structure in a first predefined relationship; and

the sensor units of the second stereoscopic sensing system are spaced apart from one another on the rigid support structure in a second predefined relationship.

12. The surgical navigation system of claim 11 , wherein the housing has an elongated shape defining a first end and an opposing second end, and wherein one of the sensor units of the first stereoscopic sensing system is disposed adjacent to the first end of the housing and the other one of the sensor units of the first stereoscopic sensing system is disposed adjacent to the opposing second end of the housing.

13. A method of operating a surgical navigation system for tracking an object within an operating room, the surgical navigation system comprising a camera unit that houses a first stereoscopic sensing system and a second stereoscopic sensing system, the first and second stereoscopic sensing systems each including sensing elements adapted to sense light, and a controller in communication with the first and second stereoscopic sensing systems, the method comprising the controller performing the following steps:

tracking the object according to a first resolution based on the first stereoscopic sensing system sensing light from the object;

tracking the object according to a second resolution that is lower than the first resolution based on the second stereoscopic sensing system sensing light from the object;

obtaining data related to the object from the second stereoscopic sensing system; and

modifying control of the sensing elements of the first stereoscopic sensing system based on the data of the object obtained by the second stereoscopic sensing system.

14. The method of claim 13 , wherein the data related to the object comprises one or more of: a type of the object; a pose of the object; a velocity of the object; an acceleration of the object; and/or an expected movement of the object.

15. The method of claim 13 , wherein modifying control of the sensing elements of the first stereoscopic sensing system comprises the controller activating a subset of the sensing elements, the subset being less than all of the sensing elements.

16. The method of claim 13 , wherein modifying control of the sensing elements of the first stereoscopic sensing system comprises the controller preventing processing of a subset of the sensing elements, the subset being less than all of the sensing elements.

17. The method of claim 13 , wherein modifying control of the sensing elements of the first stereoscopic sensing system comprises the controller adjusting a size of a field-of-view viewed by the first stereoscopic sensing system.

18. The method of claim 13 , wherein modifying control of the sensing elements of the first stereoscopic sensing system comprises the controller defining a size and/or position of a region-of-interest viewed by the first stereoscopic sensing system, the region-of-interest being a sub-region within a full field-of-view of the first stereoscopic sensing system.

19. The method of claim 13 , wherein modifying control of the sensing elements of the first stereoscopic sensing system comprises the controller adjusting a frequency by which the sensing elements are read-out.

20. A surgical navigation system for tracking an object within an operating room, the surgical navigation system comprising:

a camera unit including:

a housing;

a first stereoscopic sensing system comprising sensor units being rigidly supported by the housing and spaced apart from one another;

a second stereoscopic sensing system comprising sensor units being rigidly supported by the housing and spaced apart from one another; and

a controller in communication with the first and second stereoscopic sensing systems, and wherein the controller is configured to:

utilize the first stereoscopic sensing system to track the object according to a first resolution;

utilize the second stereoscopic sensing system to track the object according to a second resolution that is lower than the first resolution;

obtain data related to the object from the second stereoscopic sensing system; and

modify control of the first stereoscopic sensing system based on the data of the object obtained by the second stereoscopic sensing system.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2024
From: UMBDENSTOCK, EMERIC; BREISACHER, JOCHEN; HOFMANN, DAVID
To: STRYKER LEIBINGER GMBH
Reel/Frame 066948/0660 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2024
From: HOEKSTRA, PAUL; MORGAN, JONATHAN
To: MAKO SURGICAL CORP.
Reel/Frame 066948/0664 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 29, 2024
From: STRYKER LEIBINGER GMBH
To: MAKO SURGICAL CORP.
Reel/Frame 066948/0667 →
Continuity (6)
Continuation 18213360 · Jun 23, 2023
Continuation 17972625 · Oct 25, 2022
Continuation 17317191 · May 11, 2021
Continuation 16441645 · Jun 14, 2019
Provisional Application 62685470 · Jun 15, 2018
Related Publication 20240267641A1 · Aug 8, 2024
References Cited (82)
US 5295483A · Nowacki et al. · 1994 [cited by applicant]
US 5817105A · Van Der Brug · 1998 [cited by applicant]
US 6141104A · Schulz et al. · 2000 [cited by applicant]
US 6442416B1 · Schultz · 2002 [cited by applicant]
US 6792135B1 · Toyama · 2004 [cited by applicant]
US 6837432B2 · Tsikos et al. · 2005 [cited by applicant]
US 6937745B2 · Toyama · 2005 [cited by applicant]
US 7492930B2 · Leitner et al. · 2009 [cited by applicant]
US 7725162B2 · Malackowski et al. · 2010 [cited by applicant]
US 8155479B2 · Hoffman et al. · 2012 [cited by applicant]
US 8181878B2 · Nunnink et al. · 2012 [cited by applicant]
US 8244495B2 · Goldbach et al. · 2012 [cited by applicant]
US 8743222B2 · Hamalainen · 2014 [cited by applicant]
US 8792963B2 · Zhao et al. · 2014 [cited by applicant]
US 9008757B2 · Wu · 2015 [cited by applicant]
US 9119655B2 · Bowling et al. · 2015 [cited by applicant]
US 9232924B2 · Liu et al. · 2016 [cited by applicant]
US 9498231B2 · Haider et al. · 2016 [cited by applicant]
US 9513113B2 · Yang et al. · 2016 [cited by applicant]
US 9699445B2 · Hoffman et al. · 2017 [cited by applicant]
US 9707043B2 · Bozung · 2017 [cited by applicant]
US 9788906B2 · Piron et al. · 2017 [cited by applicant]
US 10013777B2 · Mariampillai et al. · 2018 [cited by applicant]
US 10038888B2 · Hoffman et al. · 2018 [cited by applicant]
US 10178368B2 · Zhao et al. · 2019 [cited by applicant]
US 11007018B2 · Breisacher et al. · 2021 [cited by applicant]
US 11510740B2 · Breisacher et al. · 2022 [cited by applicant]
US 11723726B2 · Breisacher et al. · 2023 [cited by applicant]
US 20050099601A1 · MacDougall et al. · 2005 [cited by applicant]
US 20050105101A1 · Duling et al. · 2005 [cited by applicant]
US 20070273766A1 · Wilson · 2007 [cited by applicant]
US 20080107305A1 · Vanderkooy et al. · 2008 [cited by applicant]
US 20080123820A1 · Kendrick et al. · 2008 [cited by applicant]
US 20090248036A1 · Hoffman et al. · 2009 [cited by applicant]
US 20100168763A1 · Zhao et al. · 2010 [cited by applicant]
US 20100268067A1 · Razzaque et al. · 2010 [cited by applicant]
US 20110235889A1 · Spahn · 2011 [cited by applicant]
US 20120002084A1 · Weissman et al. · 2012 [cited by applicant]
US 20130010087A1 · Nieten et al. · 2013 [cited by applicant]
US 20130335417A1 · McQueston et al. · 2013 [cited by applicant]
US 20140128881A1 · Tyc et al. · 2014 [cited by applicant]
US 20140200621A1 · Malackowski et al. · 2014 [cited by applicant]
US 20140340524A1 · Holz · 2014 [cited by applicant]
US 20150141755A1 · Tesar · 2015 [cited by applicant]
US 20150297313A1 · Reiter et al. · 2015 [cited by applicant]
US 20150332465A1 · Schmidt et al. · 2015 [cited by applicant]
US 20160275703A1 · Mariampillai et al. · 2016 [cited by applicant]
US 20160335766A1 · Ambwani et al. · 2016 [cited by applicant]
US 20170196641A1 · Jagga et al. · 2017 [cited by applicant]
US 20170214858A1 · Keller et al. · 2017 [cited by applicant]
US 20170245946A1 · Tabandeh et al. · 2017 [cited by applicant]
US 20170265949A1 · Crawford et al. · 2017 [cited by applicant]
US 20170281283A1 · Siegler et al. · 2017 [cited by applicant]
US 20170304007A1 · Piron et al. · 2017 [cited by applicant]
US 20170325897A1 · Isaacs et al. · 2017 [cited by applicant]
US 20170325898A1 · Isaacs et al. · 2017 [cited by applicant]
US 20170340406A1 · Hendriks et al. · 2017 [cited by applicant]
US 20180068441A1 · Yu et al. · 2018 [cited by applicant]
US 20180106600A1 · Greenspan et al. · 2018 [cited by applicant]
US 20180140197A1 · Wang et al. · 2018 [cited by applicant]
US 20180308263A1 · Mariampillai et al. · 2018 [cited by applicant]
US 20190261931A1 · Ross · 2019 [cited by applicant]
US 20190387149A1 · Breisacher et al. · 2019 [cited by applicant]
US 20200078113A1 · Sawhney et al. · 2020 [cited by applicant]
US 20200170718A1 · Peine · 2020 [cited by applicant]
US 20200222123A1 · Handley et al. · 2020 [cited by applicant]
US 20210052348A1 · Schwagli · 2021 [cited by applicant]
US 20210259787A1 · Breisacher et al. · 2021 [cited by applicant]
US 20220211447A1 · Calloway et al. · 2022 [cited by applicant]
US 20220345651A1 · Tachi · 2022 [cited by examiner]
US 20230034032A1 · Suzuki · 2023 [cited by examiner]
US 20230045799A1 · Breisacher et al. · 2023 [cited by applicant]
US 20230329803A1 · Breisacher et al. · 2023 [cited by applicant]
US 20240233310A1 · Yamaguchi · 2024 [cited by examiner]
WO 2016041050A1 · 2016 [cited by applicant]
WO 2017054817A1 · 2017 [cited by applicant]
WO 2017208186A2 · 2017 [cited by applicant]
WO 2018150336A1 · 2018 [cited by applicant]
WO 2018210422A1 · 2018 [cited by applicant]
WO 2019029934A1 · 2019 [cited by applicant]
Bouget, D. et al., Vision-Based and Marker-Less Surgery Tool Detection and Tracking: A Review of the Literature:, Med. Image Analysis, vol. 35, 2017, pp. 633-654. [cited by applicant]
English language abstract and machine-assisted English translation for WO 2017/054817 extracted from espacenet.com database on Jul. 15, 2019, 29 pages. [cited by applicant]