Robust packet loss handling in recording real-time video
Improved systems and methods of video decoding and recording in real-time video communications for use in lossy network environments. The disclosed systems and methods can employ a plurality of wait time thresholds for retransmission of missing video packets, based at least on the processing performed on the respective video packets, such processing including video decoding in a real-time video communication between client devices, and video recording and storing in a video file. The disclosed system and methods can also adaptively perform error concealment on video frames in the bitstream domain prior to recording and storing encoded video frame data in a video file, based at least on estimates of the complexities of the respective video frames.
1. A method of video decoding and recording in real-time video communications in a lossy network environment, comprising:
obtaining, at a video receiver, one or more video packets of a real-time video communication, the video receiver including a video decoder for decoding data of at least one video frame, and a video recorder for recording data of the at least one video frame;
determining whether the one or more obtained video packets of the real-time video communication have one or more missing video packets, the one or more missing video packets being indicative of a possible eventual packet loss at the video receiver;
having determined that the one or more obtained video packets of the real-time video communication have one or more missing video packets, transmitting, by the video receiver, at least one retransmission request for at least some of the missing video packets;
monitoring a time elapsed since transmission of the at least one retransmission request;
in the event the monitored time elapsed exceeds a first wait time threshold based on a first type of processing relating to the decoding of the data of the at least one video frame, determining a first occurrence of eventual packet loss relating to the decoding of the data of the at least one video frame; and
in the event the monitored time elapsed exceeds a second wait time threshold based on a second type of processing relating to the recording of the data of the at least one video frame, determining a second occurrence of eventual packet loss relating to the recording of the data of the at least one video frame.
2. The method of claim 1 further comprising:
having determined the first occurrence of eventual packet loss relating to decoding the data of the at least one video frame, transmitting, by the video receiver, at least one request for an intra-coded video frame; and
directing the video decoder to at least temporarily stop the decoding of the data until receipt of the requested intra-coded video frame at the video receiver.
3. The method of claim 2 further comprising:
upon the receipt of the requested intra-coded video frame, directing the video decoder to resume the decoding of the data of the at least one video frame in a real-time fashion.
4. The method of claim 1 further comprising:
having determined the second occurrence of eventual packet loss relating to recording the data of the at least one video frame, transmitting, by the video receiver, at least one request for an intra-coded video frame; and
directing the video recorder to at least temporarily stop the recording of the data until receipt of the requested intra-coded video frame at the video receiver.
5. The method of claim 4 further comprising:
upon the receipt of the requested intra-coded video frame, directing the video recorder to resume the recording of the data of the at least one video frame in a non-real-time fashion.
6. The method of claim 1 wherein the second wait time threshold is greater than the first wait time threshold.
7. The method of claim 1 wherein the transmitting of the at least one retransmission request includes transmitting at least one Generic Negative Acknowledgement (GNACK) message.
8. The method of claim 1 wherein the transmitting of the at least one request for the intra-coded video frame includes transmitting at least one Picture Loss Indication (PLI) message.
9. The method of claim 1 further comprising:
in the event the one or more obtained video packets of the real-time video communication do not have one or more missing video packets, reconstructing data of at least one encoded video frame from the one or more video packets, and decoding the data of the at least one encoded video frame.
10. The method of claim 9 further comprising:
providing the at least one video frame in a video sequence for viewing on a display of a client device, or for re-encoding for transmission of re-encoded data to another client device.
11. The method of claim 1 further comprising:
in the event the obtained video packets of the real-time video communication do not have one or more missing video packets, recording, by the video recorder, the data of the at least one video frame in a video file.