Decoding regions of interest in computer game video first while concealing missing parts using multiple decoders

1 . An apparatus comprising:

at least one processor assembly configured to:

encode a first portion of a video using a first encoder of the at least one processor assembly, the first portion of the video including a region of interest (ROI) portion of the video;

concurrently encode a second portion of the video using a second encoder of the at least one processor assembly, the second portion of the video including a non-ROI portion of the video; and

schedule transmission such that the encoded first portion is sent over a network to at least one receiver before sending the encoded second portion, the encoded first portion being prioritized in transmission over the network based on a latency target thereby facilitating:

responsive to the second portion being unavailable within the latency target and prior to presentation, update of a frame buffer to (i) obfuscate the second portion, (ii) reuse prior frame content, or (iii) reconstruct missing second portion content using a machine learning model; and

responsive to the second portion becoming available, blending of the non-ROI portion with the ROI portion.

2 . The apparatus of claim 1 , wherein the video comprises a computer game video.

3 . The apparatus of claim 1 , wherein the ROI portion is identified using gaze tracking at the receiver.

4 . The apparatus of claim 1 , wherein the ROI portion is identified by a source of the video.

5 . The apparatus of claim 1 , wherein the ROI portion is identified with the machine learning model.

6 . The apparatus of claim 1 , wherein the processor assembly is configured to send to the receiver an indication of portions of the video that were not able to be encoded.

7 . The apparatus of claim 1 , wherein the at least one processor assembly is configured to transmit, with the first portion, an indication identifying non-ROI sub-regions not yet encoded and reconstruction instructions for use by the receiver during the latency target.

8 . The apparatus of claim 1 , wherein the at least one processor assembly is configured to adapt the ROI based on gaze tracking reported from the receiver and switches encoder assignment in real time without interrupting the scheduled transmission of the encoded first portion.

9 . An apparatus comprising:

at least one processor assembly including a frame buffer, the at least one processor assembly configured to:

decode a first portion of a video using a first decoder of the at least one processor assembly into the frame buffer, the first portion of the video including a region of interest (ROI) portion of the video;

concurrently decode a second portion of the video using a second decoder of the at least one processor assembly into the frame buffer, the second portion of the video including a non-ROI portion of the video;

present the decoded first portion on at least one video display while the second portion is being decoded;

responsive to the second portion being unavailable within a latency period and prior to presentation, update the frame buffer to (i) obfuscate the second portion, (ii) reuse prior frame content, or (iii) reconstruct missing second portion content using a machine learning model; and

responsive to the second portion becoming available, blending the non-ROI portion with the ROI portion.

10 . The apparatus of claim 9 , wherein the ROI portion is identified using gaze tracking.

11 . The apparatus of claim 9 , wherein the ROI portion is identified by a source of the video.

12 . The apparatus of claim 9 , wherein the ROI portion is identified with the machine learning model.

13 . The apparatus of claim 9 , wherein responsive to the latency period elapsing, the at least one processor assembly is configured to reuse non-ROI content from a previous frame updated by motion vectors while the ROI for a current frame is displayed.

14 . The apparatus of claim 9 , wherein responsive to the latency period elapsing, the at least one processor assembly is configured to a trained model to reconstruct missing second portion content and blends the reconstructed content with the decoded first portion content.

15 . A method, comprising:

transmitting a region of interest (ROI) portion of a video frame to a receiver before transmitting a portion of the frame outside the ROI portion;

decoding the ROI portion of the video with a decoder of the receiver;

presenting the ROI portion on a video display regardless of whether portions the portion of the frame outside the ROI portion is decoded for presentation;

responsive to the portion of the frame outside the ROI portion being unavailable within a latency period and prior to presentation, updating a frame buffer to (i) obfuscate the portion of the frame outside the ROI portion, (ii) reuse prior frame content, or (iii) reconstruct missing portion of the frame outside the ROI portion using a machine learning model; and

responsive to the portion of the frame outside the ROI portion becoming available, blending a non-ROI portion with the ROI portion.

16 . The method of claim 15 , comprising using plural encoders to encode the frame.

17 . The method of claim 15 , comprising using plural decoders to decode the frame.

18 . The method of claim 15 , comprising reconstructing the portion of the frame outside the ROI portion at the receiver using the machine learning model.

19 . The method of claim 15 , further comprising signaling, with the transmitted ROI portion, a mask identifying non-ROI areas to be concealed until corresponding data arrives or is reconstructed.

20 . The method of claim 15 , wherein the blending comprises placing portions of the ROI portion into adjacent non-ROI areas to improve visual continuity when the unavailable portions of the frame outside the ROI become available.