IP Library › Granted Patent US 12,640,273
Granted Patent B2
US 12,640,273 · App. 17/332,480 · Granted May 26, 2026

Dynamic adaptation system for surgical simulation

Inventors: Frederick E Shelton, IV (Hillsboro, OH); Charles J. Scheib (Loveland, OH); Jason L. Harris (Lebanon, OH); Kevin M. Fiebig (Cincinnati, OH)
Assignee: Cilag GmbH International
G16H50/50A61B18/14A61B34/25A61B34/30A61B34/37A61B90/36G06F30/20G06N20/00G06N20/10G09B9/00G09B19/003G09B23/30G16H15/00G16H20/40G16H30/40G16H40/20G16H40/63G16H40/67G16H50/70G16H70/20A61B2018/1253A61B2018/126A61B34/10A61B2034/101A61B2034/102A61B2034/105A61B2034/107A61B2034/252A61B2034/254A61B2034/256A61B2034/258A61B2090/363A61B2090/365A61B2218/002A61B2218/008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,640,273
App. No.
17/332,480
Granted
May 26, 2026
Kind
B2
Abstract

Systems, methods, and instrumentalities for managing surgical consequences in a simulated surgical task may be provided. A simulation environment to simulate a surgical task defined by a set of environmental parameters may be received. Surgical interaction data may be received. In examples, the surgical interaction data may comprise an indication of a triggering event and/or an indication of consequence environmental parameters. The surgical interaction data may be associated with a surgical task. The consequence environmental parameters may be associated with the triggering event. User interaction data may be received. The user interaction data may be based on a user action within the simulation environment and may be associated with a surgical task. On a condition that the received user interaction data matches the triggering event, the simulation environment may be modified, for example, based on the consequence environmental parameters.

Claims (47)

1 . A device for managing surgical consequences in a simulated surgical task, the device comprising:

a display;

a user input system; and

a processor communicatively coupled with the display and the user input system, the processor configured to:

display on the display a simulation environment to simulate a surgical task defined by a set of environmental parameters;

receive, from a surgical data system, surgical interaction data, wherein the surgical interaction data comprises an indication of a triggering event, a probability percentage associated with the triggering event, and an indication of consequence environmental parameters associated with the triggering event, wherein the triggering event is associated with a surgical action and a designated force applied to a body part during the surgical action, and wherein the surgical interaction data is based on real-world data sets compiled from previously performed surgical procedures;

receive, from the user input system, user interaction data based on a user action within the simulation environment and associated with the surgical task, the user interaction data comprising a simulated action, a simulated force associated with the simulated action, and a remaining time to complete the surgical task;

determine the user interaction data matches the triggering event, wherein the user interaction data matches the triggering event on a condition that the simulated action corresponds to the surgical action and the simulated force equals the designated force; and

on a condition that the user interaction data matches the triggering event, that the simulation environment has been modified in response to a triggering event a percentage of times that is less than the probability percentage, that an experience level associated with an operator of the user input system meets a defined level, and that the remaining time to complete the surgical task exceeds a threshold value, modify on the display the simulation environment based on the consequence environmental parameters, wherein the set of environmental parameters are associated with a baseline anatomy and the consequence environmental parameters are associated with a modified anatomy.

2 . The device of claim 1 , wherein the surgical interaction data is associated with the surgical task and the surgical task comprises a medical procedure and a medical procedure context.

3 . The device of claim 2 , wherein the triggering event and the consequence environmental parameters are based on historical data associated with the medical procedure and the medical procedure context, wherein the historical data is based on previously performed live procedures recorded by a surgical hub.

4 . The device of claim 3 , wherein the historical data is local data associated with a medical facility, wherein the previously performed live procedures are performed by medical staff associated with the medical facility, wherein the surgical hub is controlled by the medical facility.

5 . The device of claim 2 , wherein the processor is further configured to:

filter the surgical interaction data to filtered surgical interaction data, wherein the filtered surgical interaction data comprises an indication of a filtered triggering event and an indication of filtered consequence environment parameters, wherein the filtered triggering event is based on the medical procedure and the medical procedure context.

6 . The device of claim 2 , wherein the processor is further configured to:

determine surgical event interaction criteria based on the surgical task; and

on a condition that the received user interaction data satisfies the surgical event interaction criteria, modify the simulation environment based on the consequence environmental parameters.

7 . The device of claim 6 , wherein the surgical event interaction criteria is associated with the medical procedure and the medical procedure context.

8 . The device of claim 2 , wherein the medical procedure context comprises at least one of: tissue friability, tissue fragility, blood flow, tissue perfusion, allergic reactions, blood pressure, or heart rate.

9 . The device of claim 1 , wherein the probability percentage is determined based on historical data.

10 . The device of claim 1 , wherein the threshold value is comprised in the surgical interaction data.

11 . The device of claim 1 , wherein the modified anatomy is a result of a surgical interaction represented by the surgical interaction data.

12 . A method performed by a device for managing surgical consequences in a simulated task, the method comprising:

displaying on a display a simulation environment to simulate a surgical task defined by a set of environmental parameters;

receiving, from a surgical data systems, surgical interaction data, wherein the surgical interaction data comprises an indication of a triggering event, a probability percentage associated with the triggering event, and an indication of consequence environmental parameters associated with the triggering event, wherein the triggering event is associated with a surgical action and a designated force applied to a body part during the surgical action, and wherein the surgical interaction data is based on real-world data sets compiled from previously performed surgical procedures;

receiving, from a user input system, user interaction data based on a user action within the simulation environment and associated with the surgical task, the user interaction data comprising a simulated action, a simulated force associated with the simulated action, and a remaining time to complete the surgical task;

determining the user interaction data matches the triggering event, wherein the user interaction data matches the triggering event on a condition that the simulated action corresponds to the surgical action and the simulated force equals the designated force; and

on a condition that the user interaction data matches the triggering event, that the simulation environment has been modified in response to a triggering event a percentage of times that is less than the probability percentage, that an experience level associated with an operator of the user input system meets a defined level, and that the remaining time to complete the surgical task exceeds a threshold value, modifying on the display the simulation environment based on the consequence environmental parameters, wherein the set of environmental parameters are associated with a baseline anatomy and the consequence environmental parameters are associated with a modified anatomy.

13 . The method of claim 12 , wherein the surgical interaction data is associated with the surgical task and the surgical task comprises a medical procedure and a medical procedure context.

14 . The method of claim 13 , wherein the triggering event and the consequence environmental parameters are based on historical data associated with the medical procedure and the medical procedure context, wherein the historical data is based on previously performed live procedures recorded by a surgical hub.

15 . The method of claim 14 , wherein the historical data is local data associated with a medical facility, wherein the previously performed live procedures are performed by medical staff associated with the medical facility, wherein the surgical hub is controlled by the medical facility.

16 . The method of claim 13 , further comprising:

filtering the surgical interaction data to filtered surgical interaction data, wherein the filtered surgical interaction data comprises an indication of a filtered triggering event and an indication of filtered consequence environment parameters, wherein the filtered triggering event is based on the medical procedure and the medical procedure context.

17 . The method of claim 13 , further comprising:

determining surgical event interaction criteria based on the surgical task; and

on a condition that the received user interaction data satisfies the surgical event interaction criteria, modifying the simulation environment based on the consequence environmental parameters.

18 . The method of claim 13 , wherein the medical procedure context comprises at least one of: tissue friability, tissue fragility, blood flow, tissue perfusion, allergic reactions, blood pressure, or heart rate.

19 . The method of claim 12 , wherein the modified anatomy is a result of a surgical interaction represented by the surgical interaction data.

20 . A system for managing surgical consequences in a simulated surgical task, the system comprising:

a display;

a user input system; and

a processor communicatively coupled with the display and the user input system, the processor configured to:

receive a simulation environment to simulate a surgical task defined by a set of environmental parameters;

receive, from a surgical data system, surgical interaction data from a surgical data system, wherein the surgical interaction data comprises an indication of a triggering event, a probability percentage associated with the triggering event, and an indication of consequence environmental parameters associated with the triggering event, wherein the triggering event is associated with a surgical action and a designated force applied to a body part during the surgical action, and wherein the surgical interaction data is based on real-world data sets compiled from previously performed surgical procedures;

receive, from the user input system, user interaction data based on a user action within the simulation environment and associated with the surgical task, the user interaction data comprising a simulated action, a simulated force associated with the simulated action, and a remaining time to complete the surgical task;

determine whether the user interaction data matches the triggering event, wherein the user interaction data matches the triggering event on a condition that the simulated action corresponds to the surgical action and the simulated force equals the designated force;

on a condition that the user interaction data matches the triggering event, that the simulation environment has been modified in response to a triggering event a percentage of times that is less than the probability percentage, that an experience level associated with an operator of the user input system meets a defined level, and that the remaining time to complete the surgical task exceeds a threshold value, modify on the display the simulation environment based on the consequence environmental parameters.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2021
From: SHELTON, FREDERICK E., IV; SCHEIB, CHARLES J.; HARRIS, JASON L.; FIEBIG, KEVIN
To: CILAG GMBH INTERNATIONAL
Reel/Frame 056634/0119 →
Continuity (2)
Provisional Application 63191681 · May 21, 2021
Related Publication 20220370135A1 · Nov 24, 2022
References Cited (99)
US 8311791B1 · Avisar · 2012 [cited by applicant]
US 9104791B2 · Cohen et al. · 2015 [cited by applicant]
US 9734632B2 · Thomas et al. · 2017 [cited by applicant]
US 9747493B2 · Rodriguez et al. · 2017 [cited by applicant]
US 9875339B2 · Namer Yelin · 2018 [cited by examiner]
US 9881520B2 · Ullrich et al. · 2018 [cited by applicant]
US 10172676B2 · Ecabert et al. · 2019 [cited by applicant]
US 20050196737A1 · Mann · 2005 [cited by applicant]
US 20080235052A1 · Node-Langlois et al. · 2008 [cited by applicant]
US 20090089092A1 · Johnson et al. · 2009 [cited by applicant]
US 20090112538A1 · Anderson et al. · 2009 [cited by applicant]
US 20090177454A1 · Bronstein et al. · 2009 [cited by applicant]
US 20090202972A1 · Adhami et al. · 2009 [cited by applicant]
US 20100178644A1 · Meglan et al. · 2010 [cited by applicant]
US 20120016691A1 · Sievenpiper et al. · 2012 [cited by applicant]
US 20120197619A1 · Namer et al. · 2012 [cited by applicant]
US 20130060146A1 · Yang et al. · 2013 [cited by applicant]
US 20140272866A1 · Kim · 2014 [cited by applicant]
US 20150005622A1 · Zhao et al. · 2015 [cited by applicant]
US 20150088547A1 · Balram et al. · 2015 [cited by applicant]
US 20160314710A1 · Jarc et al. · 2016 [cited by applicant]
US 20170148213A1 · Thomas et al. · 2017 [cited by applicant]
US 20170181808A1 · Panescu et al. · 2017 [cited by applicant]
US 20170319283A1 · Suresh et al. · 2017 [cited by applicant]
US 20180060455A1 · Castillo · 2018 [cited by applicant]
US 20180098813A1 · Nesichi et al. · 2018 [cited by applicant]
US 20180116724A1 · Gmeiner et al. · 2018 [cited by applicant]
US 20180189966A1 · Kamen et al. · 2018 [cited by applicant]
US 20180247558A1 · Livneh · 2018 [cited by applicant]
US 20180344308A1 · Nawana et al. · 2018 [cited by applicant]
US 20180348876A1 · Banerjee et al. · 2018 [cited by applicant]
US 20190000578A1 · Yu et al. · 2019 [cited by applicant]
US 20190059997A1 · Frushour · 2019 [cited by applicant]
US 20190125361A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190200844A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201118A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201137A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201140A1 · Yates et al. · 2019 [cited by applicant]
US 20190206564A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190206569A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190206576A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190272917A1 · Couture et al. · 2019 [cited by applicant]
US 20190325574A1 · Jin et al. · 2019 [cited by applicant]
US 20190362651A1 · Barral et al. · 2019 [cited by applicant]
US 20190380792A1 · Poltaretskyi et al. · 2019 [cited by applicant]
US 20200118691A1 · Kiljanek · 2020 [cited by applicant]
US 20200174451A1 · Chanin · 2020 [cited by applicant]
US 20200237452A1 · Wolf et al. · 2020 [cited by applicant]
US 20200242686A1 · García Giraldez et al. · 2020 [cited by applicant]
US 20200273581A1 · Wolf et al. · 2020 [cited by applicant]
US 20200275976A1 · Mckinnon et al. · 2020 [cited by applicant]
US 20200279506A1 · Mignan et al. · 2020 [cited by applicant]
US 20200405403A1 · Shelton, IV et al. · 2020 [cited by applicant]
US 20210137634A1 · Lang · 2021 [cited by applicant]
US 20210142497A1 · Pugh et al. · 2021 [cited by applicant]
US 20220233119A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220370131A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220370132A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220370133A1 · Scheib et al. · 2022 [cited by applicant]
US 20220370134A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220370136A1 · Scheib et al. · 2022 [cited by applicant]
US 20220370137A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220370138A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220375570A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220375620A1 · Scheib et al. · 2022 [cited by applicant]
US 20220384022A1 · Matsuura et al. · 2022 [cited by applicant]
US 20230293236A1 · Wright et al. · 2023 [cited by applicant]
CN 110638529 · 2020 [cited by applicant]
CN 110638529A · 2020 [cited by applicant]
DE 102015208804A1 · 2016 [cited by applicant]
EP 2269693A1 · 2011 [cited by applicant]
EP 3367387A1 · 2018 [cited by applicant]
EP 3506290A1 · 2019 [cited by applicant]
EP 3649994A1 · 2020 [cited by applicant]
JP 2021509034A · 2021 [cited by applicant]
KR 101940706 · 2019 [cited by applicant]
KR 101940706B1 · 2019 [cited by applicant]
WO 2011108994A1 · 2011 [cited by applicant]
WO 2017027638A1 · 2017 [cited by applicant]
WO 2020072255A1 · 2020 [cited by applicant]
WO 2020163358A1 · 2020 [cited by applicant]
Andersen, Daniel et al., “Augmented Visual Instruction for Surgical Practice and Training”, IEEE Workshop on Virtual and Augmented Realities for Good; Reutlingen, Germany, Mar. 18, 2018, 5 pages. [cited by applicant]
Elhelw, Mohamed A. , “Overview of Surgical Simulation”, Center for Informatics Science; Nile University; Giza, Egypt, May 2020, 26 pages. [cited by applicant]
Reiter, Austin et al., “Surgical Structured Light for 3D Minimally Invasive Surgical Imaging”, 2014 IEEE/RSJ International Conference on Intelligent Robot and Systems; Chicago, IL, Sep. 14-18, 2014, 6 pages. [cited by applicant]
Wang, Yang et al., “High Resolution Acquisition, Learning, and Transfer of Dynamic 3-D Facial Expressions”, EuroGraphics 2004; vol. 23, No. 3, 2004, 10 pages. [cited by applicant]
Gallagher, A. et al., “Fundamentals of Surgical Simulation”, Gallagher, et al., “Fundamentals of Surgical Simulation, Principles and Practices Improving Medical Outcome—Zero Tolerance”, DOI 10.1007/978-0-85729-763-1_12,… [cited by applicant]
“International Search Report and Written Opinion received for PCT Application No. PCT/IB2022/054724”, Jul. 5, 2022, 13 pages. [cited by applicant]
“International Search Report and Written Opinion received for PCT Application No. PCT/IB2022/054738”, Jul. 14, 2022, 14 pages. [cited by applicant]
“International Search Report and Written Opinion received for PCT Application No. PCT/IB2022/054739”, Jul. 18, 2022, 13 pages. [cited by applicant]
“International Search Report and Written Opinion received for PCT Application No. PCT/IB2022/054725”, Jul. 8, 2022, 11 pages. [cited by applicant]
Bell, Ron “Introduction to IEC 61508”, Measurement + Control vol. 42/6, Jul. 2009, pp. 174-179. [cited by applicant]
Ben-Hamadou, et al., “Construction of Extended 3D Field of Views of the Internal Bladder Wall Surface: A Proof of Concept”, 3D Research, 3D Display Research Center, Seoul, vol. 7, No. 19, Jun. 15, 2016, pp. 1-23. [cited by applicant]
Bernhardt, et al., “The Status of Augmented Reality in Laparoscopic Surgery as of 2016”, Medical Image Analysis, vol. 37, 2017, pp. 66-90. [cited by applicant]
ISO 26262-1, “Road Vehicles—Functional Safety—Part 1: Vocabulary”, International Organization for Standardization, Edition 2, Dec. 2018, 13 pages. [cited by applicant]
JP H01-185876 (A), Cited in Japanese Office Action dated Feb. 3, 2026 in related matter number END9338JPPCT6. [cited by applicant]
JP H11-197159 (A), Cited in Japanese Office Action dated Feb. 3, 2026 in related matter number END9338JPPCT8. [cited by applicant]
JP 2005-124824 (A), Cited in Japanese Office Action dated Jan. 27, 2026 in related matter number END9338JPPCT3. [cited by applicant]
JP 2006-149560 (A), Cited in Japanese Office Action dated Feb. 3, 2026 in related matter number END9338JPPCT10. [cited by applicant]
JP 2008-080021 (A), Cited in Japanese Office Action dated Feb. 3, 2026 in related matter number END9338JPPCT10. [cited by applicant]