Apparatus, systems, and methods for intraoperative visualization
Errors in a blended stream that would result in non-display or obscuring of a live video stream from a medical device may be automatically detected, and a failover stream corresponding to the first live video stream may be displayed to medical personnel. For example, one or more second input streams that are being blended may contain no data or invalid data which may result in the blended stream not displaying (or obscuring) the live video stream (if the blended were displayed). Switching from blending to a failover buffer may occur within the time to process a single video image frame. Upon detection (prior to display) that the blended stream would not display the live video stream, display of the live video stream from the failover buffer may be initiated. Other aspects are also described and claimed.
1. A processor-implemented method comprising:
receiving, one or more graphics streams with an alpha channel, wherein the graphics streams comprise one or more graphics frames;
receiving at least one first intraoperative live video stream during an intraoperative procedure, the first intraoperative live video stream comprising a plurality of live video frames, the at least one first intraoperative live video stream being received from a medical device;
storing at least one current graphics frame into a first buffer;
blending the at least one current graphics frame from the first buffer with at least one current live video frame of the plurality of live video frames into at least one blended frame; and
initiating, in real-time during the intraoperative procedure, display of the at least one current live video frame, in response to a contemporaneous determination that the blending would result in non-display of the at least one current live video frame.
2. The method of claim 1 , wherein the determination to initiate display of the current live video frame occurs within the time to complete the blending of the at least one current graphics frame with the at least one current live video frame.
3. The method of claim 1 , further comprising:
displaying, in real time, the at least one current live video frame along with an error indication.
4. The method of claim 1 , wherein the determination to initiate display of the at least one current live video frame is based on a ratio of a number of solid lines in the at least one blended frame to a total number of lines in the at least one blended frame.
5. The method of claim 4 , wherein the determination to initiate display of the at least one current live video frame is made when the ratio exceeds a threshold.
6. The method of claim 1 , further comprising:
initiating display of the at least one blended frame, in response to a determination that the blending would result in display of the at least one current live video frame.
7. The method of claim 6 , wherein the determination to initiate display of the at least one blended frame is made when a ratio of a number of solid lines in the at least one blended frame to a total number of lines in the at least one blended frame does not exceed a threshold.
8. The method of claim 1 , wherein receiving at least one first intraoperative live video stream comprises at least one of:
optionally storing the at least one current live video frame into at least one second buffer; or
downsampling the at least one current live video frame and storing the at least one downsampled current live video frame into at least one third buffer; or
a combination thereof.
9. The method of claim 8 , wherein the downsampling is performed in real-time.
10. The method of claim 1 , wherein the at least one first intraoperative live video stream is obtained by mixing two intraoperative live video streams.
11. The method of claim 1 , wherein the at least one first intraoperative live video stream comprises a stereoscopic live video stream.
12. The method of claim 1 performed by a processor, wherein the processor is coupled to memory in which a plurality of buffers including the first buffer are implemented.
13. A system comprising:
a video interface to receive at least one first intraoperative live video stream from a medical device during an intraoperative procedure, wherein the at least one first intraoperative live video stream comprises a plurality of live video frames;
a graphics interface to receive, one or more graphics streams with an alpha channel, wherein the graphics streams comprise one or more graphics frames;
a storage comprising a plurality of buffers; and
a processor configured to
store at least one current graphics frame, of the one or more graphics frames, into a first buffer of the plurality of buffers;
blend the at least one current graphics frame from the first buffer with at least one current live video frame of the plurality of live video frames into at least one blended frame, and
initiating, in real-time during the intraoperative procedure, display of the at least one current live video frame, in response to a determination that the blending would result in non-display of the at least one current live video frame.
14. The system of claim 13 , wherein a determination to initiate display of the at least one current live video frame occurs within the time to complete the blending of the at least one current graphics frame with the at least one current live video frame.
15. The system of claim 13 , further comprising the processor causing the display of the at least one current live video frame along with an error indication.
16. The system of claim 13 , wherein a determination to initiate display of the at least one current live video frame is based on a ratio of a number of solid lines in the at least one blended frame to a total number of lines in the at least one blended frame.
17. The system of claim 16 , wherein the determination to initiate display of the at least one current live video frame is made when the ratio exceeds a threshold.
18. The system of claim 13 , wherein the processor is further configured to
initiate display of the at least one blended frame, in response to a determination that the blending would result in display of the at least one current live video frame.
19. The system of claim 18 , wherein the determination to initiate display of the at least one blended frame is made when a ratio of a number of solid lines in the at least one blended frame to a total number of lines in the at least one blended frame does not exceed a threshold.
20. A non-transitory computer-readable medium comprising instructions to configure a processor to
receive a graphics stream that comprises a plurality of graphics frames;
receive an intraoperative live video stream comprising a plurality of live video frames from a medical device during an intraoperative procedure;
store a current graphics frame from the graphics stream into a buffer;
blend the current graphics frame from the buffer with a current live video frame of the intraoperative live video stream, into a blended frame; and
initiate in real-time during the intraoperative procedure, display of the current live video frame, in response to a contemporaneous determination that the blending would result in non-display of the current live video frame.