Operating room remote monitoring
OR computing system monitoring is described including self-monitoring and monitoring, with or without intervention, via a monitoring computing system. For a patient procedure in an OR, positions of one or more objects are tracked using sensors. Intra-operative data including pose information is communicated (e.g. in real time) to a monitoring computing system to present the intra-operative data including object positions in a GUI. Pose information may be sensor data with which to calculate a pose of an object or pre-calculated pose data. Intra-operative data may be a workflow state of the procedure to display workflow UI screens. A virtual view of the OR may be presented from pose information and geometric data for OR equipment. Working volumes of OR equipment including collision warning may be presented. OR equipment may include a localizer and/or a robot. Self-monitoring may evaluate progress in a procedure and trigger a request (e.g. to monitor).
1 . A distributed computing system to monitor a computer assisted procedure performed in an operating room (OR), the computing system comprising:
a monitoring computing system located externally to the OR and comprising:
at least one first processor coupled to each of:
a first data store comprising a first memory or other first storage storing first instructions;
a first display device; and
a first communication system to communicate via a network with an OR computing system located within the OR; and
the OR computing system, the OR computing system comprising:
at least one second processor coupled to each of:
a second data store comprising a second memory or other second storage storing second instructions;
a second display device; and
a second communication system to communicate via the network with the monitoring computing system; and
wherein:
the second instructions are executable by the at least one second processor to configure the at least one second processor to assist an OR user to perform the computer-assisted procedure relative to a patient in the OR by:
tracking respective positions of one or more objects during the computer-assisted procedure using sensor data representing the respective positions of the one or more objects;
communicating intra-operative data including pose information for the one or more objects as tracked to the monitoring computing system via the network for passively monitoring the computer assisted procedure in the OR without intervening in the performing of the computer assisted procedure;
the first instructions are executable by the at least one first processor to configure the at least one first processor to monitor the computer assisted procedure performed by the OR user using the OR computing system by:
receiving via the network from the OR computing system the intra-operative data including pose information for the one or more objects as tracked; and
present, to a user of the monitoring computing system and in a graphical user interface (GUI) via the first display device, at least some of the intra-operative data including the respective positions of the one or more objects using the pose information and geometrical data for the one or more objects for passively monitoring the computer-assisted procedure without intervening;
the pose information comprises one or both of: sensor data from sensors located in the OR and coupled directly or indirectly to the OR computing system to calculate a pose of an object; or pose data representing the pose of the object as calculated by the OR computing system using the sensor data.
2 . The distributed computing system of claim 1 wherein the computer-assisted procedure comprises one of: performing a computer-assisted localization to track the one or more objects; or controlling a positioning of at least one of the one or more objects using a robot.
3 . The distributed computing system of claim 1 wherein the sensor data comprises one or both of: camera measurements from a camera sensor of an optical localization system; or pose data calculated using the camera measurements.
4 . The distributed computing system of claim 1 wherein the sensor data comprises a motor encoder measurement of motors of a robot controlling the positioning of at least one of the one or more objects; and wherein the pose information comprises one or both of: the motor encoder measurement; or pose data calculated using the motor encoder measurement.
5 . The distributed computing system of claim 1 wherein the monitoring computing system further comprises at least one of a video camera and a microphone to capture monitoring video and/or voice data of the user of the monitoring computing system and wherein the first instructions configure the at least one first processor to communicate the monitoring video and/or voice data to the OR computing system for presenting within the OR and to receive video and/or voice data from the OR user via the OR computing system to establish a voice and/or voice and video communication link between the OR user and the user of the monitoring system to communicate to help the OR user to perform the computer-assisted procedure.
6 . The distributed computing system of claim 1 wherein the first instructions are further executable to configure the at least one first processor to:
receive one of:
a) a workflow state from the OR computing system representing a progress of the OR user in a workflow comprising UI screens for the computer-assisted procedure; or
b) a mirror of a display device of the OR computing system sharing the UI screens in accordance with the progress of the OR user in the workflow for the computer-assisted procedure; and
present the UI screens of the workflow via the first display device to the user of the monitoring computing system to passively monitor the computer-assisted procedure.
7 . The distributed computing system of claim 1 wherein the first instructions are further executable to configure the at least one first processor to:
receive input to define intervention data to take over control of the OR computing system from the OR user and communicate the intervention data to the OR computing system, wherein the input comprises user input from the user of the monitoring computing system who is distinct from the OR user located in the OR, the intervention data enabling the user of the monitoring computing system to intervene for help in the computer-assisted procedure; and
receive from the OR computing system and present to user of the monitoring computing system further intra-operative data including further pose information for the one or more objects as further tracked in response to the intervention by the monitoring computing system.
8 . The distributed computing system of claim 1 wherein the first instructions configure the at least one first processor to:
obtain the geometrical data comprising 3D data models for rendering 3D objects in a user interface; and
use the positional information and the respective data models to render a 3D view of at least the one or more objects in the GUI.
9 . The distributed computing system of claim 1 wherein the OR computing system comprises or is coupled to components of one of a localization system and a robotic surgery system and wherein the first instructions configure the monitoring computing system to access geometrical data representing at least one of the components to present a rendering of the at least one of the components in a virtual view showing a layout of OR equipment of the OR, wherein the at least one of the components are distinct from the one or more objects as tracked for the computer-assisted procedure.
10 . The distributed computing system of claim 9 wherein the first instructions are executable to further configure the at least one first processor to provide a control selected from a pan control to pan the virtual view, a tilt control to tilt the virtual view, a zoom control to zoom in and/or out the virtual view, and a working volume control to display a working volume for at least one component of OR equipment in the virtual view.
11 . The distributed computing system of claim 1 wherein the first instructions are executable to further configure the at least one first processor to receive a message and log data from the OR computing system to initiate monitoring by the monitoring computing system to provide help for the procedure, wherein the log data provides a representation of a progress of the computer-assisted procedure prior to the initiation of monitoring for presenting by the monitoring computing system.
12 . The distributed computing system of claim 1 , wherein the first instructions are executable to further configure the at least one first processor to selectively monitor a plurality of respective OR computing systems each of the plurality of OR computing systems actively performing respective computer-assisted procedures, wherein a user interface is configured to receive input to select one of the respective computer-assisted procedures or one of the plurality of OR computing systems to invoke the monitoring.
13 . A computer implemented method to receive monitoring by a remotely located monitoring computing system, the method for a distributed computing system comprising an operating room (OR) computing system located in an OR and the remotely located monitoring computing system located externally to the OR, the method comprising:
by the OR computing system:
performing a computer-assisted procedure relative to a patient, tracking respective positions of one or more objects during the computer-assisted procedure in the OR in which sensor data, representing the respective positions of the one or more objects within the OR, is received by the OR computing system from sensors associated with the one or more objects; and
generating intra-operative data for the procedure including pose information for the one or more objects as tracked;
presenting intra-operative data via a display device coupled to the OR computing system in the OR for at least one OR user, the at least one OR user located in the OR;
communicating the intra-operative data via an external communication network to the remotely located monitoring computing system for monitoring the OR for passively monitoring the computer assisted procedure in the OR without intervening in the performing of the computer assisted procedure;
by the remotely located monitoring computing system:
receiving and presenting, to at least one monitoring user distinct from the at least one OR user, the intra-operative data in a graphical user interface (GUI) comprising the respective positions of one or more of the objects to passively monitor the computer-assisted procedure without intervening;
wherein the pose information comprises one or both of:
the sensor data with which to calculate a pose of an object; and
pose data representing the pose of the object as calculated by the OR computing system using the sensor data.
14 . The method of claim 13 comprising, by the OR computing system, self-monitoring a progression of the computer-assisted procedure and, responsive to the progression, communicating a message to the monitoring computing system to invoke a remote monitoring of the OR computing system.
15 . The method of claim 13 wherein the self monitoring comprises:
storing log data with which to monitor a progress of the computer-assisted procedure;
monitoring the progress of the computer-assisted procedure using the log data;
determining a measure of progress responsive to the monitoring; and
responsive to the measure of progress, communicate a message identifying the measure of progress to the remotely located monitoring computing system to monitor the OR computing system.
16 . The method of claim 15 wherein the measure of progress is: (i) a failure to advance the state of a workflow from one state to a next state; (ii) a repetition of one state of the workflow; or (iii) responsive to an expiry of a threshold period of time to advance the state of the workflow from one state to a next state.
17 . The method of claim 15 comprising, by the OR computing system:
communicating the message during the computer-assisted procedure;
receiving monitoring data from the monitoring computing system during the computer-assisted procedure; and
utilizing the monitoring data from the remotely located monitoring computing system via the OR computing system, wherein the monitoring data from the remotely located monitoring computing system is one or both of: i) intervention data to at least partially control the OR computing system; and ii) audio and/or video data to present via an audio and/or display device.
18 . The method of claim 13 further comprising, by the monitoring computer system:
receiving input to define intervention data to take over control of the OR computing system from the OR user, wherein the input comprises user input from the user of the monitoring computing system, the intervention data enabling the user of the monitoring computing system to intervene for help in the computer-assisted procedure;
communicating the intervention data to the OR computing system and
receiving from the OR computing system and presenting to user of the monitoring computing system further intra-operative data including further pose information for the one or more objects as further tracked in response to the intervention by the monitoring computing system.
19 . The method of claim 13 , wherein the OR computing system comprises or is coupled to components of one of a localization system and a robotic surgery system, and whether the method further comprises, by the monitoring computer system:
accessing geometrical data representing at least one of the components to present a rendering of the at least one of the components in a virtual view showing a layout of OR equipment of the OR, wherein the at least one of the components are distinct from the one or more objects as tracked for the computer-assisted procedure; and
providing a control selected from a pan control to pan the virtual view, a tilt control to tilt the virtual view, a zoom control to zoom in and/or out the virtual view, and a working volume control to display a working volume for at least one component of OR equipment in the virtual view.
20 . The method of claim 13 , wherein the monitoring computer system is coupled to a plurality of OR computing systems in respective ORs, and the method further comprises, by the monitoring computer system:
selectively monitoring the plurality of respective OR computing systems, each of the plurality of OR computing systems actively performing respective computer-assisted procedures, wherein a user interface of the monitoring computer system is configured to receive input to select one of the respective computer-assisted procedures or one of the plurality of OR computing systems to invoke the monitoring.