Granted Patent B2

US 12,193,766 · App. 17/212,001 · Granted Jan 14, 2025

Inventors: Frederick E. Shelton, IV (Hillsboro, OH); Jason L. Harris (Lebanon, OH); Chester O. Baxter, III (Loveland, OH); Mark S. Zeiner (Mason, OH)

Assignee: Cilag GmbH International

A61B34/25A61B18/00A61B34/20A61B34/35A61B34/37A61B90/30A61B90/361G06F3/017G06T7/10G06V40/20G16H20/40G16H40/63A61B2017/00203A61B2017/00207A61B2017/00221A61B2018/00994A61B2018/1253A61B2018/126A61B2034/2051A61B2034/2055A61B2034/2063A61B2034/2065A61B2034/2074A61B2034/252A61B2034/254A61B2034/256A61B2090/373A61B2090/3954A61B2218/002A61B2218/007A61B2218/008G06T2207/20081G06V2201/034

Abstract

Claims (81)

1. A situationally aware surgical system configured for use during a surgical procedure performed on a patient by a surgical staff in an operating room, the situationally aware surgical system comprising:

a surgical instrument;

an imaging system configured to capture an image of the operating room and generate a image data based on the captured image of the operating room; and

a cloud-based analytics subsystem configured to be communicably coupled to the surgical instrument and the imaging system, wherein the cloud-based analytics subsystem comprises a control circuit configured to:

receive the image data from the imaging system;

detect a target in the operating room based on the image data received from the imaging system;

generate a feature vector based on a machine learning model;

determine a characteristic of the surgical procedure based on the feature vector and the detected target; and

automatically control the surgical instrument based on the determined characteristic of the surgical procedure.

2. The situationally aware surgical system of claim 1 , wherein the cloud-based analytics subsystem further comprises memory configured to store an edge detection algorithm and a template of the target, and wherein the control circuit is further configured to:

generate an outline of the target based on an edge detection algorithm stored in the memory; and

compare the generated outline of the target to the template of the target stored in the memory, wherein the detection of the target in the operating room is further based on the comparison of the generated outlines to the template stored in the memory.

3. The situationally aware surgical system of claim 1 , wherein cloud-based analytics subsystem further comprises memory configured to store the machine learning model, and wherein the control circuit is further configured to:

compare the generated feature vector to the detected target, and wherein the control of the surgical instrument is further based on the comparison of the generated feature vector to the detected target.

4. The situationally aware surgical system of claim 3 , wherein the generated feature vector comprises at least one of: an edge, a curvature, and a corner, or combinations thereof.

5. The situationally aware surgical system of claim 1 , wherein the determined characteristic comprises a physical property of the detected target, an action taken by the detected target, or an interaction between the detected target and a detected second target, or a combination thereof.

6. The situationally aware surgical system of claim 1 , wherein the operating room comprises a sterile barrier that defines a sterile field and a non-sterile field.

7. The situationally aware surgical system of claim 6 , wherein the determined characteristic comprises a position of the target relative to the sterile barrier.

8. The situationally aware surgical system of claim 1 , wherein the detected target comprises a member of the surgical staff.

9. The situationally aware surgical system of claim 8 , wherein the determined characteristic comprises:

a gesture performed by the member of the surgical staff;

a surgical instrument held by the member of the surgical staff;

a location of the member of the surgical staff;

a detected total number of members of the surgical staff;

an interaction between the member of the surgical staff and the surgical instrument;

a passing of the surgical instrument from the member of the surgical staff to a detected second member of the surgical staff; or

a physical property of the member of the surgical staff; or

a combination thereof.

10. The situationally aware surgical system of claim 9 , wherein the physical property of the member of the surgical staff comprises:

a posture;

an arm position; or

a wrist angle; or

a combination thereof.

11. The situationally aware surgical system of claim 1 , wherein the detected target comprises the surgical instrument.

12. The situationally aware surgical system of claim 11 , wherein the determined characteristic comprises:

a pose of the surgical instrument;

an operational status of the surgical instrument; or

a position of the surgical instrument relative to the patient; or

a combination thereof.

13. The situationally aware surgical system of claim 1 , wherein controlling the surgical instrument comprises:

activating the surgical instrument;

changing an operational mode of the surgical instrument; or

modifying operational parameters of the surgical instrument; or

a combination thereof.

14. A cloud-based analytics system for use during a surgical procedure performed on a patient by a surgical staff in an operating room, wherein the cloud-based analytics system is configured to be communicably coupled to a surgical instrument and an imaging system, the cloud-based analytics system comprising:

a control circuit configured to:

receive image data associated with a captured image of the operating room from the imaging system;

detect a target in the operating room based on the image data received from the imaging system;

generate a feature vector based on a machine learning model;

determine a characteristic of the surgical procedure based on the feature vector and the detected target; and

automatically control the surgical instrument based on the determined characteristic of the surgical procedure.

15. The cloud-based analytics system of claim 14 , further comprising a memory configured to store an edge detection algorithm and a template of the target, and wherein the control circuit is further configured to:

generate an outline of the target based on an edge detection algorithm stored in the memory; and

compare the generated outline of the target to the template of the target stored in the memory, wherein the detection of the target in the operating room is further based on the comparison of the generated outlines to the template stored in the memory.

16. The cloud-based analytics system of claim 14 , further comprising a memory configured to store the machine learning model, and wherein the control circuit is further configured to:

compare the generated feature vector to the detected target, and wherein the control of the surgical instrument is further based on the comparison of the generated feature vector to the detected target.

17. The cloud-based analytics system of claim 16 , wherein the detected target comprises a member of the surgical staff, and wherein the determined characteristic comprises:

a gesture performed by the member of the surgical staff;

a surgical instrument held by the member of the surgical staff;

a location of the member of the surgical staff;

a detected total number of members of the surgical staff;

an interaction between the member of the surgical staff and the surgical instrument;

a passing of the surgical instrument from the member of the surgical staff to a detected second member of the surgical staff; or

a physical property of the member of the surgical staff; or

a combination thereof.

18. The cloud-based analytics system of claim 16 , wherein the detected target comprises the surgical instrument, and wherein the determined characteristic comprises:

a pose of the surgical instrument,

an operational status of the surgical instrument; or

a position of the surgical instrument relative to the patient; or

a combination thereof.

19. A method of controlling a surgical instrument during a surgical procedure performed on a patient by a surgical staff in an operating room, wherein the surgical procedure involves a surgical instrument and an imaging system, the method comprising:

receiving, via a control circuit, image data associated with a captured image of the operating room from the imaging system;

detecting, via the control circuit, a target in the operating room based on the image data received from the imaging system;

generating, via the control circuit, a feature vector based, at least in part, on a machine learning model stored in a memory;

determining, via the control circuit, a characteristic of the surgical procedure based on the feature vector and the detected target; and

automatically controlling, via the control circuit, the surgical instrument based on the determined characteristic of the surgical procedure.

20. The method of claim 19 , further comprising:

generating an outline of the target based on an edge detection algorithm stored in a memory; and

comparing, via the control circuit, the generated outline of the target to a template of the target stored in the memory, wherein the detection of the target in the operating room is further based on the comparison of the generated outlines to the template stored in the memory.

21. The method of claim 19 , further comprising:

comparing, via the control circuit, the generated feature vector to the detected target, and wherein the control of the surgical instrument is further based on the comparison of the generated feature vector to the detected target.

Assignments (2)

Continuity (16)

References Cited (400)