IP Library › Granted Patent US 12682578
Granted Patent B2
US 12682578 · App. 17/688,633 · Granted Jul 14, 2026

Systems and methods for changing display overlay of surgical field view based on triggering events

Inventors: Frederick E. Shelton, IV (Hillsboro, OH); Jacqueline C. Aronhalt (Loveland, OH); Shane R. Adams (Lebanon, OH); Cory G. Kimball (Hamilton, OH); Monica L. Z. Rivard (Cincinnati, OH); Leonardo N. Rossoni (Rahway, NJ); Matthew D. Cowperthwait (Cincinnati, OH); Risto Kojcev (Santa Clara, CA); Felix J. Bork (Schnürpflingen, DE)
Assignee: Cilag GmbH International
G06T11/00A61B34/20A61B34/25A61B34/30A61B34/32A61B34/76A61B90/36A61B90/361A61B90/37A61B90/39G06F3/14G06F3/1454G06F3/147G06T7/0012G06T7/20G06T19/006G06V20/20G08B21/182G16H20/40G16H40/67H04L63/105H04L67/12H04W24/10H04W76/14A61B2034/102A61B2034/2055A61B2034/2072A61B2034/252A61B2034/254A61B2090/365A61B2090/368A61B2090/371A61B2090/372A61B2090/373A61B2090/3937A61B2090/3975G06T2207/10028G06T2207/30024G06T2210/41
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12682578
App. No.
17/688,633
Granted
Jul 14, 2026
Kind
B2
Abstract

A surgical system is disclosed including an imaging device, a display configured to show a livestream of a surgical field of a surgical procedure, and a control system operably coupled to the imaging device and the display. The livestream is captured by the imaging device. The control system is configured to overlay, on the livestream, information associated with the surgical procedure, detect an occurrence of a triggering event, and adjust the overlaid information based on the occurrence of the triggering event.

Claims (49)

1 . A surgical system, comprising:

an imaging device;

a display configured to show a livestream of a surgical field of a surgical procedure, wherein the livestream is captured by the imaging device; and

a control system operably coupled to the imaging device and the display, wherein the control system is configured to:

overlay, on the livestream, information associated with the surgical procedure;

set a triggering event count representing a number of detected partial triggering events within the surgical field;

detect a plurality of partial triggering events, wherein the plurality of partial triggering events comprise a negative triggering event;

determine an amount by which to decrement the triggering event count based on the negative triggering event;

adjust the triggering event count, based at least in part on the amount by which to decrement the triggering event count, by incrementing or decrementing the triggering event count based on the detected plurality of partial triggering events; and

detect the occurrence of the triggering event based on detecting the triggering event count reaching or exceeding a triggering event threshold.

2 . The surgical system of claim 1 , wherein control system is configured to monitor a parameter associated with a patient to which the surgical procedure is being performed, wherein the triggering event comprises a value of the parameter reaching or exceeding a parameter threshold.

3 . The surgical system of claim 1 , wherein control circuit is configured to monitor a parameter associated with a patient to which the surgical procedure is being performed, wherein the triggering event comprises a rate of change of a value of the parameter reaching or exceeding a rate of change threshold.

4 . The surgical system of claim 1 , further comprising a surgical device associated with a surgeon, wherein the triggering event comprises an event induced by the surgical device.

5 . The surgical system of claim 1 , further comprising:

a first surgical device associated with a surgeon; and

a second surgical device unassociated with the surgeon;

wherein the triggering event comprises an event induced by the second surgical device.

6 . The surgical system of claim 1 , wherein the partial triggering events further comprise additive triggering events, and wherein adjusting the triggering event count, based at least in part on the amount by which to decrement the triggering event count, comprises:

adding to the triggering event count based on an occurrence of an additive triggering event; and

subtracting from the triggering event count based on an occurrence of a negative triggering event.

7 . The surgical system of claim 1 , further comprising a memory storing data associated with objects, wherein the control system is further configured to:

identify an object in the livestream; and

compare the object to the data associated with objects stored in the memory;

wherein the triggering event comprises the object being identified from the comparison.

8 . The surgical system of claim 1 , further comprising an interactive sensor, wherein the triggering event comprises a user interacting with the interactive sensor.

9 . The surgical system of claim 1 , further comprising a surgical instrument comprising an end effector, wherein the control system is further configured to monitor a position of the end effector, and wherein the triggering event comprises the end effector reaching a location.

10 . The surgical system of claim 1 , wherein the control system is configured to monitor a position of a user, and wherein the triggering event comprises the user reaching a location.

11 . The surgical system of claim 10 , wherein adjusting the overlaid information comprises overlaying information associated with the location.

12 . The surgical system of claim 11 , wherein the overlaid information associated with the location comprises information associated with the surgical procedure.

13 . The surgical system of claim 11 , wherein the control system is further configured to determine a step of the surgical procedure, and wherein the overlaid information associated with the location comprises information associated with the step of the surgical procedure.

14 . The surgical system of claim 11 , wherein the overlaid information associated with the location comprises information based on user input.

15 . The surgical system of claim 1 , wherein the control system is further configured to:

determine a step of the surgical procedure; and determine a component required to perform the step of the surgical procedure; wherein the triggering event comprises the control system determining the component required to perform the step of the surgical procedure; and

wherein adjusting the overlaid information comprises providing directional information for where the component can be located.

16 . The surgical system of claim 1 , wherein the ray comprises a first display, wherein the surgical system further comprises a second display, and wherein adjusting the overlaid information comprises moving overlaid information from the first display to the second display.

17 . A surgical system, comprising:

an imaging device;

a display configured to show a livestream of a surgical field of a surgical procedure, wherein the livestream is captured by the imaging device; and

a control system operably coupled to the imaging device, and the display wherein the control system is configured to:

overlay, on the livestream, information associated with the surgical procedure;

determine a step of the surgical procedure;

set a triggering event count representing a number of detected partial triggering events within the surgical field;

detect a plurality of partial triggering events, wherein the plurality of partial triggering events comprise a negative triggering event;

determine an amount by which to decrement the triggering event count based on the negative triggering event;

adjust the triggering event count, based at least in part on the amount by which to decrement the triggering event count, by incrementing or decrementing the triggering event count based on the detected plurality of partial triggering events;

detect the occurrence of the triggering event based on detecting the triggering event count reaching or exceeding a triggering event threshold.

18 . The surgical system of claim 17 , wherein the partial triggering events comprise additive triggering events, and wherein adjusting the triggering event count, based at least in part on the amount by which to decrement the triggering event count, comprises:

adding to the triggering event count based on an occurrence of an additive triggering event; and

subtracting from the triggering event count based on an occurrence of a negative triggering event.