Granted Patent B2

US 12,533,008 · App. 17/330,949 · Granted Jan 27, 2026

Inventors: Steen MøLler Hansen (Los Gatos, CA); Henriette Schultz Kirkegaard (Copenhagen V, DK)

Assignee: Cilag GmbH International

A61B1/0005A61B1/00193A61B1/00194A61B1/04A61B1/3132A61B90/37G01B11/24G06T15/08G06T19/20G06T7/0012G06T7/344G06T7/70G06T15/205

Abstract

Claims (37)

1 . A system, comprising:

at least one data processor; and

memory storing instructions configured to cause the at least one data processor to perform operations comprising:

receiving three-dimensional (3D) data characterizing a target surgical surface;

receiving real-time surgical tool spatial position data characterizing a location of a portion of a surgical tool;

determining a 3D surface contour of the target surgical surface based on the received 3D data;

determining a real-time distance of a surgical tool from the target surgical surface based on the received 3D data and the received real-time surgical tool spatial position data;

determining depiction data characterizing a graphical representation that indicates the determined real-time distance of the surgical tool relative to the target surgical surface, the graphical representation overlaid onto an image including the determined 3D surface contour, the graphical representation configured to shrink when the real-time distance decreases and configured to expand when the real-time distance increases; and

providing the depiction data to a display unit for real-time correlated depiction of movements of the surgical tool.

2 . The system of claim 1 , further comprising a 3D surface sensor system configured to acquire the 3D data.

3 . The system of claim 2 , wherein the 3D surface sensor system comprises at least one local reference sensor.

4 . The system of claim 2 , wherein the 3D surface sensor system comprises a 3D optical sensor system and comprising an optical source and an optical reader, wherein the optical source is configured to emit an optical tracking light pattern, such that the pattern, when impinged onto and reflected and/or scattered from the surface and acquired by the optical reader, reveals a contour of the target surgical surface, and wherein the 3D surface contour is based on the revealed contour.

5 . The system of claim 4 , further comprising an endoscope, and wherein at least one of the optical source and the optical reader is positioned on an endoscope.

6 . The system of claim 1 , further comprising a position sensor configured to acquire real-time surgical tool spatial position data, the position sensor comprising at least one of a 3D optical sensor, an acoustic sensor, a magnetic sensor, an electric sensor, an accelerometer, a gyroscope, a gravimeter, an inertial navigation system, and a local positioning system.

7 . The system of claim 1 , wherein the operations further comprise:

determining a real time spatial orientation of the surgical tool using spatial orientation data characterizing a spatial orientation of at least a part of the surgical tool;

calculating orientation depiction data representing a depiction of the real time spatial orientation of the surgical tool onto the 3D surface contour, and

transmitting the orientation depiction data to the display unit for display thereon.

8 . The system of claim 7 , further comprising an orientation sensor configured to acquire the spatial orientation data.

9 . The system of claim 1 , wherein the depiction data comprises data encoding a depiction of a dynamic pattern representation, and wherein the dynamic pattern representation comprises a depiction of a virtual pattern of an emitted light pattern impinged onto the determined 3D surface contour.

10 . The system of claim 9 , wherein the dynamic pattern representation comprises a depiction of the virtual pattern onto the determined 3D surface contour, such that the depiction provides a dynamic modification of the virtual pattern wherein the dynamic modification is correlated to a determined real-time spatial position of the surgical tool.

11 . The system of claim 1 , wherein the depiction data comprises a depiction of a dynamic augmented reality representation, wherein the dynamic augmented reality representation comprises an augmented reality representation of a determined real-time spatial position of the surgical tool relative to the determined 3D surface contour, and wherein the augmented reality representation is dynamically modified based on changes of a spatial position and an orientation of the surgical tool relative to the determined 3D surface contour.

12 . A method, comprising:

receiving three-dimensional (3D) data characterizing a target surgical surface;

receiving real-time surgical tool spatial position data characterizing a location of a portion of a surgical tool;

determining a 3D surface contour of the target surgical surface based on the received 3D data;

determining a real-time distance of a surgical tool from the target surgical surface based on the received 3D data and the received real-time surgical tool spatial position data;

determining depiction data characterizing a graphical representation that indicates the determined real-time distance of the surgical tool relative to the target surgical surface, the graphical representation overlaid onto an image including the determined 3D surface contour, the graphical representation configured to shrink when the real-time distance decreases and configured to expand when the real-time distance increases; and

providing the depiction data to a display unit for real-time correlated depiction of movements of the surgical tool.

13 . The method of claim 12 , further comprising acquiring the 3D data.

14 . The method of claim 12 , further comprising:

determining a real time spatial orientation of the surgical tool using spatial orientation data characterizing a spatial orientation of at least a part of the surgical tool;

calculating orientation depiction data representing a depiction of the real time spatial orientation of the surgical tool onto the 3D surface contour, and

transmitting the orientation depiction data to the display unit for display thereon.

15 . The method of claim 12 , wherein the depiction data comprises data encoding a depiction of a dynamic pattern representation, and wherein the dynamic pattern representation comprises a depiction of a virtual pattern of an emitted light pattern impinged onto the determined 3D surface contour.

16 . The method of claim 15 , wherein the dynamic pattern representation comprises a depiction of the virtual pattern onto the determined 3D surface contour, such that the depiction provides a dynamic modification of the virtual pattern wherein the dynamic modification is correlated to a determined real-time spatial position of the surgical tool.

17 . The method of claim 12 , wherein the depiction data comprises a depiction of a dynamic augmented reality representation, wherein the dynamic augmented reality representation comprises an augmented reality representation of a determined real-time spatial position of the surgical tool relative to the determined 3D surface contour, and wherein the augmented reality representation is dynamically modified based on changes of a spatial position and an orientation of the surgical tool relative to the determined 3D surface contour.

Assignments (4)

Continuity (3)

References Cited (400)