Adaptation and adjustability or overlaid instrument information for surgical systems
An augmented reality display system used during a surgical procedure is disclosed. The system includes an imaging device to capture a real image of a surgical area during the surgical procedure. An augmented reality display presents functional overlay data associated with critical operations of a surgical instrument being actively visualized and interactions of the surgical instrument with tissue in the surgical area. The functional data is overlaid onto the real image of the surgical area. The functional data overlay is a combination of aspects of the critical operations of the surgical instrument and the interaction of the surgical instrument with the tissue in the surgical area. A processor receives functional data from the surgical instrument, determines the overlaid data related to the functional aspect of the surgical instrument, and combines the aspect of the tissue in the surgical area with the functional data received from the surgical instrument.
1 . A system, comprising:
an imaging device to capture a real image of a surgical area during the surgical procedure;
an augmented reality display to present a functional data overlay associated with a surgical instrument;
an energy generator coupled to the surgical instrument, wherein the surgical instrument employs radio frequency (RF) energy and ultrasonic energy during a surgical procedure, wherein the surgical instrument is configured to provide hemostasis to tissue of the surgical area;
a surgical hub coupled to the energy generator and to the augmented reality display, wherein the surgical hub is configured to:
provide a live feed of the surgical area to the augmented reality display to display the live feed of the surgical area;
generate overlay information associated with operations of the surgical instrument and interactions of the surgical instrument with tissue of the surgical area;
detect a surgical parameter associated with the surgical procedure; and
adjust the overlay information based on the detected surgical parameter,
wherein the augmented reality display is configured to display a view of the surgical area, the surgical instrument, and a panel overlay to display the adjusted overlay information.
2 . The system of claim 1 , wherein the augmented reality display shows an end effector of the surgical instrument grasping tissue and a panel overlay displaying case information, systems notifications, or device panels, or any combination thereof, overlaid over the live feed of the surgical area.
3 . The system of claim 2 , wherein a location, opacity, size, and placement of the panel overlay is customizable.
4 . The system of claim 3 , wherein the panel overlay is configured to be turned on or off individually or turned on/off as a group.
5 . The system of claim 2 , wherein the panel overlay comprises at least one of data input information from capital equipment, generators, insufflator, smoke evacuator, electronic health record, laparoscope, computer, surgical devices, wired and wirelessly connected devices, surgeon profile preferences that may be saved, recalled or edited, or any combination thereof.
6 . The system of claim 2 , wherein the panel overlay comprises case information including at least one of Patient Name, Surgeon Name, Case Time, or Instrument Activations, or combinations thereof.
7 . The system of claim 2 , wherein the panel overlay comprises system notifications including at least one of connect instrument status, minor error alert, medium error alert, or major error alert, or any combination thereof.
8 . The system of claim 2 , wherein the panel overlay comprises information associated with the surgical instrument connected to the system to provide advanced hemostasis.
9 . The system of claim 2 , wherein the panel overlay comprises a visible patient panel overlay.
10 . The system of claim 2 , wherein the panel overlay comprises a device panel overlay comprising at least one of device name, device settings, or device supplemental features, or any combination thereof.
11 . The system of claim 2 , wherein the panel overlay comprises a plurality of panel overlays in a stacked configuration.
12 . The system of claim 2 , wherein the panel overlay comprises a plurality of panel overlays in an expanded configuration.
13 . The system of claim 12 , wherein the panel overlay is configured to change dynamically to show state changes such as device activation or power level adjustment.
14 . The system of claim 2 , wherein the panel overlay depicts optimal device performance (ODP) guide images or other instructions for use (IFU)/informational sources.
15 . The system of claim 1 , wherein the intraoperative data display comprises a secondary configurable panel.
16 . The system of claim 15 , wherein the secondary configurable panel changes dynamically based on the selected customized laparoscopic overlay fields displayed in the surgical field of view of a live surgical feed area of the intraoperative data display.
17 . The system of claim 16 , wherein the customized laparoscopic overlay fields comprise at least one of a bottom edge panel, a top left corner panel, a top center panel, or a side edge panel, or any combination thereof.
18 . The system of claim 2 , wherein the panel overlay displays device troubleshooting information.
19 . The system of claim 2 , wherein the panel overlay displays at least one of alerts, warnings, device information, or device features, or any combination thereof.