IP Library › Granted Patent US 12,482,555
Granted Patent B2
US 12,482,555 · App. 17/384,348 · Granted Nov 25, 2025

Surgical data system and classification

Inventors: Frederick E. Shelton, IV (Hillsboro, OH); Shane R. Adams (Lebanon, OH); Bernhard Adolf Fuerst (Sunnyvale, CA)
Assignee: Cilag GmbH International
G16H40/20A61B17/00A61B18/1206A61B34/10A61B34/20A61B34/25A61B34/30A61B34/32A61B90/08A61B90/37G05B13/0265G06F3/14G06F3/1423G06F3/167G06F9/4881G06F9/542G06F13/4068G06F16/211G06F16/284G06F16/285G06N20/00G06Q10/30G06T11/60G08B5/22G10L15/22G16H10/60G16H15/00G16H20/40G16H30/40G16H40/40G16H40/63G16H40/67G16H50/20G16H50/70H04L1/22H04L41/12H04L65/80H04L67/12H04L67/125H04N5/272H04N7/15A61B8/06A61B2017/00221A61B2018/00702A61B2018/00994A61B2034/2072A61B2034/254A61B2090/364A61B2090/365A61B2090/373G06F21/6245G06F40/169G10L2015/223G16H30/20H02J7/0063
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,482,555
App. No.
17/384,348
Granted
Nov 25, 2025
Kind
B2
Abstract

A device may classify data associated with a surgical event of a surgery. The device may generate a third surgical data stream based on a mode of interaction between a first surgical data stream and a second surgical data stream. The device may determine a third classification parameter for the third surgical data stream based on a first classification parameter associated with the first surgical data stream, a second classification parameter associated with the second surgical data stream, and the mode of interaction between the first surgical data stream and the second surgical data stream. The device may determine a data handling scheme for the third surgical data stream based on the third classification parameter in consistency with a healthcare data policy. The device may handle the third surgical data stream according to the data handling scheme.

Claims (51)

1 . A surgical hub configured to classify surgical data, comprising:

a processor configured to:

receive a first stream of surgical data via a first surgical data interface of the surgical hub, wherein the first surgical data interface is configured to receive the first stream from a first surgical instrument;

receive a second stream of surgical data via a second surgical data interface of the surgical hub, wherein the second surgical data interface is configured to receive the second stream from a second surgical instrument;

determine, at the surgical hub, a first classification parameter associated with the first stream based on a field of a data packet in the first stream, wherein the first classification parameter indicates a first priority level and a first privacy level associated with the first stream;

determine, at the surgical hub, a second classification parameter associated with the second stream based on a field of a data packet in the second stream, wherein the second classification parameter indicates a second priority level and a second privacy level associated with the second stream;

identify, at the surgical hub, a surgical event associated with at least one of the first stream or the second stream;

determine, at the surgical hub, a mode of interaction between the first stream and the second stream based on the identified surgical event, wherein the mode of interaction indicates how the first stream is to be combined, enhanced, or modified using the second stream to improve a surgical outcome;

generate a third stream of surgical data, at the surgical hub, wherein the third stream comprises surgical information derived by applying the mode of interaction between the first stream and the second stream;

determine, at the surgical hub, a third classification parameter for the third stream based on the first classification parameter associated with the first stream, the second classification parameter associated with the second stream, and the mode of interaction between the first stream and the second stream, wherein the third classification parameter indicates a third priority level and a third privacy level associated with the third stream;

determine, from a plurality of communication channels, a communication channel associated with the surgical hub based on a security level of the communication channel and the third classification parameter that is determined at the surgical hub;

encrypt the third stream according to the security level of the communication channel;

establish the communication channel; and

send the encrypted third stream via the communication channel.

2 . The surgical hub of claim 1 , wherein the processor is further configured to generate situational data associated with the surgical event based on the mode of interaction, wherein the third stream of surgical data is generated based on the situational data.

3 . The surgical hub of claim 1 , wherein the mode of interaction comprises one or more of an enrichment of the first stream of surgical data using the second stream of surgical data, an aggregation of the first stream and the second stream, or a synthesis of the first stream and the second stream.

4 . The surgical hub of claim 1 , wherein the processor is further configured to:

identify the first surgical data interface, wherein the determination of the first classification parameter is based on the identified first surgical data interface, and the first surgical data interface is designated to communicate with a first type of surgical instrument; and

identify the second surgical data interface, wherein the determination of the second classification parameter is based on the identified second surgical data interface, and the second surgical data interface is designated to communicate with a second type of surgical instrument.

5 . The surgical hub of claim 1 , wherein the determination of the first classification parameter is based on decoding the first classification parameter in the first stream of surgical data, and the determination of the second classification parameter is based on decoding the second classification parameter in the second stream of surgical data.

6 . The surgical hub of claim 1 , wherein the determination of the first classification parameter comprises decoding the first stream of surgical data and inferring the first classification parameter based on the decoded first stream, and wherein the determination of the second classification parameter comprises decoding the second stream of surgical data and inferring the second classification parameter based on the decoded second stream.

7 . The surgical hub of claim 1 , wherein at least one of the first classification parameter, the second classification parameter, or the third classification parameter is multidimensional.

8 . The surgical hub of claim 1 , wherein the third classification parameter indicates one or more of a content type of the third stream, a context of the third stream, a retention period associated with the third stream, or a user preference associated with the third stream.

9 . The surgical hub of claim 1 , wherein the processor is further configured to determine a data handling scheme based on the third classification parameter, and the data handling scheme indicates one or more of a type of storage location for the third stream of surgical data or a reliability level associated with the communication channel used for sending the third stream.

10 . The surgical hub of claim 1 , wherein the processor is further configured to:

determine, based on the third classification parameter, that the third stream of surgical data has the highest classification level among a plurality of streams of surgical data that are to be transmitted, wherein the communication channel is associated with the least amount of interruption among transmission resources that are available to be used for the transmission of the plurality of streams; and

based on the determination that the third stream has the highest classification level among the plurality of streams to be transmitted, repeat the sending of the encrypted third stream.

11 . The surgical hub of claim 1 , wherein the processor is further configured to determine a data handling scheme for the second stream of surgical data based on the second classification parameter, the second classification parameter and the third classification parameter are the same, and the data handling scheme for the second stream is the same as a data handling scheme for the third stream of surgical data.

12 . The surgical hub of claim 1 , wherein the third classification parameter for the third stream of surgical data is determined using a surgical data classification engine tailored to solve multidimensional classification parameters.

13 . The surgical hub of claim 9 , wherein the data handling scheme is in consistency with a healthcare data policy that comprises one or more rules that are consistent with Health Insurance Portability and Accountability Act (HIIPPA).

14 . A method performed by a surgical hub for classifying surgical data, comprising:

receiving a first stream of surgical data via a first surgical data interface of the surgical hub, wherein the first surgical data interface is configured to receive the first stream from a first surgical instrument;

receiving a second stream of surgical data via a second surgical data interface of the surgical hub, wherein the second surgical data interface is configured to receive the second stream from a second surgical instrument;

determining, at the surgical hub, a first classification parameter associated with the first stream based on a field of a data packet in the first stream, wherein the first classification parameter indicates a first priority level and a first privacy level associated with the first stream;

determining, at the surgical hub, a second classification parameter associated with the second stream based on a field of a data packet in the second stream, wherein the second classification parameter indicates a second priority level and a second privacy level associated with the second stream;

identifying, at the surgical hub, a surgical event associated with at least one of the first stream or the second stream;

determining, at the surgical hub, a mode of interaction between the first stream and the second stream based on the identified surgical event, wherein the mode of interaction indicates how the first stream is to be combined, enhanced, or modified using the second stream to improve a surgical outcome;

generating a third stream of surgical data, at the surgical hub, wherein the third stream comprises surgical information derived by applying the mode of interaction between the first stream and the second stream;

determining, at the surgical hub, a third classification parameter for the third stream based on the first classification parameter associated with the first stream, the second classification parameter associated with the second stream, and the mode of interaction between the first stream and the second stream, wherein the third classification parameter indicates a third priority level and a third privacy level associated with the third stream;

determining, from a plurality of communication channels, a communication channel associated with the surgical hub based on a security level of the communication channel and the third classification parameter that is determined at the surgical hub;

encrypting the third stream according to the security level of the communication channel;

establishing the communication channel; and

sending the encrypted third stream via the communication channel.

15 . The method of claim 14 , wherein the determination of the first classification parameter comprises decoding the first stream of surgical data and inferring the first classification parameter based on the decoded first stream, and wherein the determination of the second classification parameter comprises decoding the second stream of surgical data and inferring the second classification parameter based on the decoded second stream.

16 . The method of claim 14 , wherein at least one of the first classification parameter, the second classification parameter, or the third classification parameter is multidimensional.

17 . The method of claim 14 , wherein the third classification parameter indicates one or more of a content type of the third stream, a context of the third stream, a retention period associated with the third stream, or a user preference associated with the third stream.

18 . The method of claim 14 , wherein the method further comprises determining a data handling scheme based on the third classification parameter, and the data handling scheme indicates one or more of a type of storage location for the third stream of surgical data or a reliability level associated with the communication channel used for sending the third stream.

19 . The method of claim 14 , the method further comprising:

determining, based on the third classification parameter, that the third stream of surgical data has the highest classification level among a plurality of streams of surgical data that are to be transmitted, wherein the communication channel is associated with the least amount of interruption among transmission resources that are available to be used for the transmission of the plurality of streams; and

based on the determination that the third stream has the highest classification level among the plurality of streams to be transmitted, repeating the sending of the encrypted third stream.

20 . The method of claim 14 , the method further comprising determining a data handling scheme for the second stream of surgical data based on the second classification parameter, the second classification parameter and the third classification parameter are the same, and the data handling scheme for the second stream is the same as a data handling scheme for the third stream of surgical data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 4, 2022
From: SHELTON, FREDERICK E., IV; ADAMS, SHANE R.; FUERST, BERNHARD ADOLF
To: CILAG GMBH INTERNATIONAL
Reel/Frame 059807/0239 →
Continuity (2)
Provisional Application 63224813 · Jul 22, 2021
Related Publication 20230026634A1 · Jan 26, 2023
References Cited (111)
US 6766373B1 · Beadle et al. · 2004 [cited by applicant]
US 8565073B2 · Rahman et al. · 2013 [cited by applicant]
US 8908678B1 · Mcgonigal et al. · 2014 [cited by applicant]
US 9011427B2 · Price et al. · 2015 [cited by applicant]
US 9283054B2 · Morgan et al. · 2016 [cited by applicant]
US 9345481B2 · Hall et al. · 2016 [cited by applicant]
US 11146690B2 · Minert · 2021 [cited by applicant]
US 11232868B1 · Sutherland · 2022 [cited by examiner]
US 11322248B2 · Grantcharov · 2022 [cited by examiner]
US 11564573B2 · Hirst · 2023 [cited by examiner]
US 20020000464A1 · Ramberg et al. · 2002 [cited by applicant]
US 20050210070A1 · Macneil · 2005 [cited by applicant]
US 20110276340A1 · DeBoer · 2011 [cited by examiner]
US 20140160259A1 · Blanquart et al. · 2014 [cited by applicant]
US 20140160260A1 · Blanquart et al. · 2014 [cited by applicant]
US 20140163927A1 · Molettiere et al. · 2014 [cited by applicant]
US 20140263552A1 · Hall et al. · 2014 [cited by applicant]
US 20140267655A1 · Richardson et al. · 2014 [cited by applicant]
US 20150119035A1 · Ganu et al. · 2015 [cited by applicant]
US 20150128274A1 · Giokas · 2015 [cited by applicant]
US 20170006135A1 · Siebel et al. · 2017 [cited by applicant]
US 20170085627A1 · Goldstein et al. · 2017 [cited by applicant]
US 20170296213A1 · Swensgard et al. · 2017 [cited by applicant]
US 20180122506A1 · Grantcharov et al. · 2018 [cited by applicant]
US 20180344308A1 · Nawana et al. · 2018 [cited by applicant]
US 20180360452A1 · Shelton, IV et al. · 2018 [cited by applicant]
US 20190191963A1 · Kuhn et al. · 2019 [cited by applicant]
US 20190200844A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190200906A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190200980A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190200988A1 · Shelton, IV · 2019 [cited by applicant]
US 20190201033A1 · Yates et al. · 2019 [cited by applicant]
US 20190201102A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201104A1 · Shelton, IV · 2019 [cited by examiner]
US 20190201115A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201123A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201124A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201125A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201126A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201127A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201129A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201137A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201140A1 · Yates et al. · 2019 [cited by applicant]
US 20190204201A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190205441A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190205566A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190205567A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190206216A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190206542A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190206551A1 · Yates et al. · 2019 [cited by applicant]
US 20190206556A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190206562A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190206569A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190206576A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190207773A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190207857A1 · Shelton et al. · 2019 [cited by applicant]
US 20190207911A1 · Wiener et al. · 2019 [cited by applicant]
US 20190333626A1 · Mansi et al. · 2019 [cited by applicant]
US 20210145523A1 · Xing · 2021 [cited by examiner]
US 20210205030A1 · Shelton, IV · 2021 [cited by examiner]
US 20210205031A1 · Shelton, IV · 2021 [cited by applicant]
US 20210212717A1 · Yates et al. · 2021 [cited by applicant]
US 20210313051A1 · Asselmann et al. · 2021 [cited by applicant]
US 20210319894A1 · Sobol et al. · 2021 [cited by applicant]
US 20210346094A1 · Fuerst · 2021 [cited by examiner]
US 20220020476A1 · Souissi · 2022 [cited by applicant]
US 20220020486A1 · Giataganas · 2022 [cited by examiner]
US 20220046292A1 · Nair et al. · 2022 [cited by applicant]
US 20220104713A1 · Shelton, IV · 2022 [cited by applicant]
US 20220104896A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220108789A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220233119A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220233135A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220233136A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220233151A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220233191A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220233252A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220233254A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220238216A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220240869A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220241028A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220241474A1 · Shelton, IV et al. · 2022 [cited by applicant]
US 20220303945A1 · Tsuda · 2022 [cited by applicant]
US 20230021920A1 · Shelton, IV et al. · 2023 [cited by applicant]
US 20230023083A1 · Shelton, IV et al. · 2023 [cited by applicant]
US 20230023635A1 · Shelton, IV et al. · 2023 [cited by applicant]
US 20230025061A1 · Shelton, IV et al. · 2023 [cited by applicant]
US 20230027210A1 · Shelton, IV et al. · 2023 [cited by applicant]
US 20230035775A1 · Kohada · 2023 [cited by applicant]
US 20230293236A1 · Wright · 2023 [cited by examiner]
EP 3506287A1 · 2019 [cited by applicant]
WO 2019119130A1 · 2019 [cited by applicant]
WO WO2020159978A1 · 2020 [cited by examiner]
WO WO2022195306A1 · 2022 [cited by examiner]
Hashimoto, Daniel. Artificial Intelligence in Surgery: Promises and Perils. Ann Surg 2018;268:70-76. (Year: 2018). [cited by examiner]
Kum, Sang-Uok. Encoding Multiple Depth Streams. Thesis. Cahpel Hill 2008. (Year: 2008). [cited by examiner]
Ban. Aggregating Long-Term Context for Learning laparoscopic and Robot-Assisted Surgical Workflows. 2021 IEEE International Conference on Robotics and Automation (ICRA 2021). Ban teaches a automated understanding of sur… [cited by examiner]
B. van Amsterdam, M. J. Clarkson and D. Stoyanov, “Gesture Recognition in Robotic Surgery: A Review,” in IEEE Transactions on Biomedical Engineering, vol. 68, No. 6, pp. 2021-2035, Jun. 2021, doi: 10.1109/TBME.2021.3054… [cited by examiner]
Gianpaolo Cugola and Alessandro Margara. 2012. Processing flows of information: From data stream to complex event processing. ACM Comput. Surv. 44, 3, Article 15 (Jun. 2012), 62 pages. (Year: 2012). [cited by examiner]
Namazi, Babak, Ganesh Sankaranarayanan, and Venkat Devarajan. “Automatic detection of surgical phases in laparoscopic videos.” Proceedings on the international conference in artificial intelligence (ICAI). 2018. (Year: … [cited by examiner]
Jagannath, et al., “An Analysis of Speech as a Modality for Activity Recognition during Complex Medical Teamwork”, Pervasive Computing Technologies for Healthcare, May 2018, pp. 1-10. [cited by applicant]
Renx, et al., “Multisensor Data Fusion in an Integrated Tracking System for Endoscopic Surgery”, IEEE Transactions on Information Technology in Biomedicine, vol. 16. No. 1, Jan. 2012, pp. 106-111. [cited by applicant]
ISO 26262-1, “Road Vehicles—Functional Safety—Part 1: Vocabulary”, International Organization for Standardization, Edition 2, Dec. 2018, 13 pages. [cited by applicant]
Khan, et al. “Zeqos: A New Energy and QoS-Aware Routing Protocol for Communication of Sensor Devices in Healthcare System”, International Journal of Distributed Sensor Networks, vol. 10, No. 06, Jun. 5, 2014, 24 pages. [cited by applicant]
International Search Report and Written Opinion, Received for PCT Application No. PCT/IB2022/056665, Oct. 31, 2022, 17 pages. [cited by applicant]
International Search Report and Written Opinion, Received for PCT Application No. PCT/IB2022/056666, Oct. 31, 2022, 11 pages. [cited by applicant]
International Search Report and Written Opinion, received for PCT Application No. PCT/IB2022/056669, Oct. 10, 2022, 16 pages. [cited by applicant]
Rahmani, et al., “Exploiting Smart E-Health Gateways at the Edge of Healthcare Internet-of-Things: A Fog Computing Approach”, Future Generation Computer Systems, vol. 78, No. 02, Feb. 2017, 47 pages. [cited by applicant]
Doan, et al., “Introduction”, Principles of Data Integration, Chapter 01, 2012, pp. 1-18. [cited by applicant]
Doan, et al., “Schema Matching and Mapping”, Principles of Data Integration, Chapter 05, 2012, pp. 121-160. [cited by applicant]
Guibert, et al., “CC-Fog: Toward Content-Centric Fog Networks for E-Health”, IEEE 19th International Conference on e-Health Networking, Applications and Services (Healthcom), Oct. 12-15, 2017, 5 pages. [cited by applicant]