IP Library Granted Patent US 10,828,104
Granted Patent B2
US 10,828,104 · App. 15/510,572 · Granted Nov 10, 2020

Surgical navigation system using image segmentation

Inventors: Arun Victor Jagga (Toronto, CA); Cameron Piron (Toronto, CA)
Assignee: Synaptive Medical (Barbados) Inc.
A61B34/00A61B34/20A61B90/10A61B90/39A61B2034/2055A61B2034/2057A61B2090/3983
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,828,104
App. No.
15/510,572
Granted
Nov 10, 2020
Kind
B2
Abstract

A system for tracking a piece of medical equipment intraoperatively using a collinear array of fiducial markers positioned at known fixed distances relative to each other on the medical equipment and a camera capturing an image on a single image plane. Representations of the fiducial markers are segmented from a captured image, 3D orientation and position of the medical equipment are calculated using the segmented representations, and the orientation and position of the medical equipment are tracked relative to the camera. The orientation and position of the medical equipment may be registered within a 3D virtual space. The system may be used as part of a surgical navigation system.

Claims (55)

1. A system for intraoperatively tracking at least one medical equipment within a three-dimensional virtual space in real time, the system comprising:

at least one collinear array, each at least one collinear array comprising a plurality of fiducial markers, each fiducial marker disposed at a known fixed distance relative to another fiducial marker in relation to each at least one medical equipment, each plurality of fiducial markers distinct from another plurality of fiducial markers, and each plurality of fiducial markers comprising a distinct combination of fiducial markers;

a camera for capturing an image of the at least one medical equipment on a single image plane, whereby a captured image is provided, the captured image comprising an image of at least some fiducial markers of the plurality of fiducial markers of each at least one medical equipment; and

a processor receiving input from the camera and configured by a set of instructions to:

segment a plurality of representations of the plurality of fiducial markers for each at least one medical equipment from the captured image, whereby a plurality of segmented representations are provided, the instruction to segment the plurality of representations comprising an instruction to filter at least one of frequency and wavelength;

calculate a three-dimensional orientation and a position of each at least one medical equipment using the plurality of segmented representations;

track the three-dimensional orientation and the position of each at least one medical equipment relative to the camera; and

register the three-dimensional orientation and the position of each at least one medical equipment within the three-dimensional virtual space,

wherein the plurality of fiducial markers comprises a plurality of active fiducial markers configured to emit a plurality of signals, each active fiducial marker of the plurality of active fiducial markers configured to emit a signal at a distinct frequency relative to another active fiducial marker, each active fiducial marker comprising a light-emitting optical fiber and a flashing light-emitting diode, the flashing light-emitting diode coupled with the light-emitting optical fiber, the distinct combination of fiducial markers comprising a distinct combination of frequencies to uniquely identify each at each at least one medical equipment, and the distinct frequency comprising a frequency range of at least 60 Hz to achieve real-time tracking of the plurality of fiducial markers,

whereby each at least one medical equipment is trackable in real time, and

whereby each at least one medical equipment is distinguishable from another at least one medical equipment.

2. The system of claim 1 ,

wherein the plurality of fiducial markers further comprises a plurality of passive fiducial markers,

wherein the known fixed distance is unique, and

wherein each active fiducial marker further comprises a modulated infrared emitter.

3. The system of claim 1 , wherein the processor is configured to segment the plurality of representations by segmenting the plurality of signals emitted by the plurality of active fiducial markers from the captured image.

4. The system of claim 3 , wherein the known fixed distance is unique.

5. The system of claim 1 , wherein the plurality of fiducial markers is collinearly disposed along a longitudinal axis of the medical equipment.

6. The system of claim 1 , wherein the camera comprises a monocular camera.

7. The system of claim 1 , wherein the camera comprises a video camera.

8. The system of claim 1 , wherein the medical equipment comprises a surgical instrument.

9. A surgical navigation system for intraoperatively tracking at least one medical equipment in real time, the system comprising:

a first medical equipment;

a first collinear array comprising a plurality of a first set of fiducial markers coupled with a first medical equipment of the at least one medical equipment, each fiducial marker of the first set of fiducial markers disposed at a known fixed distance relative to another fiducial marker of the first set of fiducial markers in relation to the first medical equipment, each fiducial marker distinct from another fiducial marker of the first set of fiducial markers;

a first camera for capturing an image of the first medical equipment on a single image plane, whereby a captured image is provided, the captured image comprising an image of at least some fiducial markers of the first set of fiducial markers;

a second camera for tracking a second set of fiducial markers; and

a processor receiving input from the first camera and the second camera, the processor configured by a set of instructions to:

register the second set of fiducial markers within a three-dimensional virtual space;

segment a plurality of representations of the first set of fiducial markers from the captured image, whereby a plurality of segmented representations is provided;

calculate a three-dimensional orientation and a position of the first medical equipment using the plurality of segmented representations; and

register the three-dimensional orientation and the position of the first medical equipment within the three-dimensional virtual space,

wherein the first set of fiducial markers comprises a plurality of active fiducial markers configured to emit a plurality of signals, each active fiducial marker of the plurality of active fiducial markers configured to emit a signal at a distinct frequency relative to another active fiducial marker, and each active fiducial marker comprising at least one light-emitting optical fiber and a flashing light-emitting diode, the flashing light-emitting diode coupled with the light-emitting optical fiber, the first set of fiducial markers comprising a combination of fiducial markers distinct in relation to the second set of fiducial markers, the distinct combination of the first set of fiducial markers comprising a distinct combination of frequencies to uniquely identify the first medical equipment, and the distinct frequency comprising a frequency range of at least 60 Hz to achieve real-time tracking of the plurality of fiducial markers,

whereby each at least one medical equipment is trackable in real time, and

whereby each at least one medical equipment is distinguishable from another at least one medical equipment.

10. The system of claim 9 , further comprising a second medical equipment having the second set of fiducial markers coupled thereto, the second set of fiducial markers comprising a plurality of passive fiducial markers, wherein each active fiducial marker further comprises a modulated infrared emitter.

11. The system of claim 10 , wherein the second medical equipment comprises an access port providing an access corridor for accessing a surgical site, and wherein the first medical equipment comprises a surgical instrument that is at least partially insertable into the access corridor.

12. The system of claim 9 , wherein the first medical equipment comprises a surgical instrument.

13. The system of claim 9 , wherein the first set of fiducial markers comprises a plurality of passive fiducial markers, and wherein the known fixed distance is unique.

14. The system of claim 9 , wherein segmenting the plurality of representations comprise segmenting the plurality of signals emitted by the first set of fiducial markers from the captured image.

15. The system of claim 14 wherein the known fixed distance is unique.

16. The system of claim 9 , wherein the first set of fiducial markers are-collinearly disposed along a longitudinal axis of the first medical equipment.

17. The system of claim 9 , wherein the first camera comprises a monocular camera, and wherein the second camera comprises a binocular camera.

18. The system of claim 9 , wherein the first camera comprises a video camera.

19. The system of claim 9 , wherein the second camera comprises a stereoscopic camera.

20. A method of fabricating a system for intraoperatively tracking at least one medical equipment within a three-dimensional virtual space in real time, the method comprising:

providing at least one collinear array, each at least one collinear array comprising a plurality of fiducial markers, each fiducial marker disposed at a known fixed distance relative to another fiducial marker in relation to each at least one medical equipment, each plurality of fiducial markers distinct from another plurality of fiducial markers, and each plurality of fiducial markers comprising a distinct combination of fiducial markers;

providing a camera for capturing an image of the at least one medical equipment on a single image plane, whereby a captured image is provided, the captured image comprising an image of at least some fiducial markers of the plurality of fiducial markers of each at least one medical equipment; and

providing a processor configured to receive input from the camera and configured by a set of instructions to:

segment a plurality of representations of the plurality of fiducial markers for each at least one medical equipment from the captured image, whereby a plurality of segmented representations are provided, the instruction to segment the plurality of representations comprising an instruction to filter at least one of frequency and wavelength;

calculate a three-dimensional orientation and a position of each at least one medical equipment using the plurality of segmented representations;

track the three-dimensional orientation and the position of each at least one medical equipment relative to the camera; and

register the three-dimensional orientation and the position of each at least one medical equipment within the three-dimensional virtual space,

wherein the plurality of fiducial markers comprises a plurality of active fiducial markers configured to emit a plurality of signals, each active fiducial marker of the plurality of active fiducial markers configured to emit a signal at a distinct frequency relative to another active fiducial marker, each active fiducial marker comprising a light-emitting optical fiber and a flashing light-emitting diode, the flashing light-emitting diode coupled with the light-emitting optical fiber, the distinct combination of fiducial markers comprising a distinct combination of frequencies to uniquely identify each at each at least one medical equipment, and the distinct frequency comprising a frequency range of at least 60 Hz to achieve real-time tracking of the plurality of fiducial markers,

whereby each at least one medical equipment is trackable in real time, and

whereby each at least one medical equipment is distinguishable from another at least one medical equipment.

Assignments (3)
SECURITY INTEREST Recorded Jan 7, 2021
From: SYNAPTIVE MEDICAL INC.
To: ESPRESSO CAPITAL LTD.
Reel/Frame 054922/0791 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2020
From: SYNAPTIVE MEDICAL (BARBADOS) INC.
To: SYNAPTIVE MEDICAL INC.
Reel/Frame 054297/0047 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 1, 2020
From: JAGGA, ARUN VICTOR; PIRON, CAMERON ANTHONY
To: SYNAPTIVE MEDICAL (BARBADOS) INC.
Reel/Frame 053954/0130 →
Continuity (1)
Related Publication 20170196641A1 · Jul 13, 2017
Cited By (1)
US 12,467,489