Granted Patent B2

US 12,731,206 · App. 18/048,711 · Granted Sep 8, 2026

Inventors: Kurtis P. Keller (Hillsborough, NC); Sharif A. Razzaque (Boulder, CO); Andrei State (Chapel Hill, NC); Caroline K. Green (Chapel Hill, NC); Jeremy D. Ackerman (Port Jefferson Station, NY)

Assignee: INNEROPTIC TECHNOLOGY, INC.

G06T3/16A61B1/0005A61B1/042A61B8/12A61B8/463A61B8/5238H04N7/181A61B1/313

Abstract

Claims (28)

1 . A method of providing real-time dynamic imagery of a medical procedure site, the method comprising:

receiving a first real-time dynamic video stream of a scene based on a first modality from a first medical device;

receiving a second real-time dynamic video stream of the scene based on a second modality from a second medical device;

obtaining at least one of first real-time spatial state data associated with the first medical device or second real-time spatial state data associated with the second medical device;

determining a relative orientation and position of the first medical device to the second medical device based on the at least one of the first real-time spatial state data or the second real-time spatial state data;

spatially registering the first real-time dynamic video stream and the second real-time dynamic video stream based on the determined relative orientation and position of the first medical device to the second medical device, wherein the first real-time dynamic video stream and the second real-time dynamic video stream align to form a composite representation of the scene; and

generating a composite real-time dynamic video stream of the scene from the composite representation.

2 . The method of claim 1 , wherein the first modality comprises a two dimensional modality.

3 . The method of claim 1 , wherein the first modality comprises a three dimensional modality.

4 . The method of claim 1 , wherein the second modality comprises a two dimensional modality.

5 . The method of claim 1 , wherein the second modality comprises a three-dimensional modality.

6 . The method of claim 1 , wherein the first modality comprises endoscopy, and wherein the endoscopy comprises a modality selected from a group consisting of: laparoscopy, hysteroscopy, thoracoscopy, arthroscopy, colonoscopy, bronchoscopy, cystoscopy, proctosigmoidoscopy, esophagogastroduodenoscopy, and colposcopy.

7 . The method of claim 1 , wherein the second modality comprises a modality selected from a group consisting of: ultrasonography, magnetic resonance imaging, x-ray imaging, computed tomography, and optical wavefront imaging.

8 . A system of providing real-time dynamic imagery of a medical procedure site, comprising:

a control system, wherein the control system is adapted to:

receive a first real-time dynamic video stream of a scene based on a first modality from a first medical device;

receive a second real-time dynamic video stream of the scene based on a second modality from a second medical device;

obtain at least one of first real-time spatial state data associated with the first medical device or second real-time spatial state data associated with the second medical device;

determine a relative orientation and position of the first medical device to the second medical device based on the at least one of the first real-time spatial state data or the second real-time spatial state data;

spatially register the first real-time dynamic video stream and the second real-time dynamic video stream based on the determined relative orientation and position of the first medical device to the second medical device, wherein the first real-time dynamic video stream and the second real-time dynamic video stream align to form a composite representation of the scene; and

generate a composite real-time dynamic video stream of the scene from the composite representation.

9 . A computer-readable non-transitory storage medium comprising computer-executable instructions that when executed by one or more processors, cause the one or more processors to:

receive a first real-time dynamic video stream of a scene based on a first modality from a first medical device;

receive a second real-time dynamic video stream of the scene based on a second modality from a second medical device;

obtain at least one of first real-time spatial state data associated with the first medical device or second real-time spatial state data associated with the second medical device;

determine a relative orientation and position of the first medical device to the second medical device based on the at least one of the first real-time spatial state data or the second real-time spatial state data;

spatially register the first real-time dynamic video stream and the second real-time dynamic video stream based on the determined relative orientation and position of the first medical device to the second medical device, wherein the first real-time dynamic video stream and the second real-time dynamic video stream align to form a composite representation of the scene; and

generate a composite real-time dynamic video stream of the scene from the composite representation.

Assignments (1)

Continuity (9)

References Cited (400)