EFFICIENT ERROR RECOVERY WITH INTRA-REFRESH
Systems and methodologies for concealing errors related to INTRA-frame losses in a transmitted video signal are provided herein. Systems and methodologies provided herein can refine a lost INTRA-coded frame and its subsequent INTER-coded frames using INTRA-coded macroblocks that are provided in a video bitstream in accordance with a Random INTRA Refresh (RIR) scheme. When an INTRA-frame is lost, INTRA-coded macroblocks can be used to refine neighboring INTER-coded macroblocks based on region filling, spatial interpolation, or other algorithms that are based on the strong correlation between values of adjacent pixels in a video signal. Further, motion compensation can be used to refine an INTER-coded pixel having an INTRA-coded pixel in its motion trajectory.
1 . A system for error-concealing a transmission error that causes one or more INTRA-coded frame losses in a video signal, comprising:
a frame loss detection component that detects a lost INTRA-coded frame in a video signal resulting from a transmission error; and
an error concealment component that conceals the transmission error in the lost INTRA-coded frame and one or more subsequent INTER-coded frames based at least in part on INTRA-coded macroblocks present in the one or more INTER-coded frames.
2 . The system of claim 2 , wherein the initial frame processing component initializes respective marks mapped to pixels in INTRA-coded macroblocks present in the one or more INTER-coded frames to indicate that the respective pixels are refreshed.
3 . The system of claim 2 , wherein the error concealment component further comprises a motion compensation component that determines whether respective motion trajectories for non-refreshed pixels in the one or more INTER-coded frames contain a refreshed pixel and refines respective non-refreshed pixels having motion trajectories that contain a refreshed pixel using a motion compensation algorithm.
4 . The system of claim 3 , wherein the motion compensation component replaces the respective non-refreshed pixels having motion trajectories that contain a refreshed pixel with the refreshed pixels in the respective motion trajectories and marks the replaced pixels as refreshed.
5 . The system of claim 3 , wherein the error concealment component further comprises a region filling component that receives a macroblock comprising pixels processed by the motion compensation component and fills the macroblock based on pixel values obtained from one or more neighboring macroblocks if the macroblock contains at least one non-refreshed pixel and at least one of the neighboring macroblocks contains only refreshed pixels.
6 . The system of claim 5 , wherein the region filling component fills the macroblock by replacing respective patches of pixels in the macroblock with patches of pixels obtained from a neighboring macroblock that contains only refreshed pixels.
7 . The system of claim 5 , wherein the region filling component fills the macroblock by replacing respective patches of pixels in the macroblock with respective patches of pixels generated as a weighted average of patches of pixels obtained from two or more neighboring macroblocks that contain only refreshed pixels.
8 . The system of claim 3 , wherein the error concealment component further comprises a DC coefficient refinement component that modifies a pixel processed by the motion compensation component upon a determination that the pixel is not refreshed and is within a predetermined distance of an INTRA-coded macroblock at least in part by changing a value of the pixel to a weighted average of an original value of the pixel and a DC coefficient of the INTRA-coded macroblock.
9 . The system of claim 3 , wherein the error concealment component further comprises a spatial interpolation component that receives a non-refreshed pixel and refines the non-refreshed pixel using a spatial interpolation algorithm if the non-refreshed pixel is within a predetermined distance of two or more refreshed pixels.
10 . The system of claim 9 , wherein the spatial interpolation component obtains an interpolated pixel value at least in part on respective values of two or more refreshed pixels to the non-refreshed pixel and respective distances between the refreshed pixels and the non-refreshed pixel and refines a value of the non-refreshed pixel using the interpolated pixel value.
11 . A method for error-concealing a video signal, comprising:
receiving a video signal containing a lost INTRA-frame; and
performing error concealment for the lost INTRA-frame and one or more INTER-frames following the lost INTRA-frame based at least in part on INTRA-blocks present in the one or more INTER-frames.
12 . The method of claim 11 , wherein the performing error concealment comprises:
determining whether respective blocks in the one or more INTER-frames are INTER-blocks or INTRA-blocks;
indicating that error concealment is complete for pixels in respective INTRA-blocks;
performing motion compensation for respective INTER-blocks in the one or more INTER-frames; and
performing region filling to refine selected INTER-blocks in the one or more INTER-frames.
13 . The method of claim 12 , wherein the performing motion compensation comprises:
determining whether respective motion trajectories for pixels in the respective INTER-blocks contain a pixel for which error concealment is complete;
replacing pixels having motion trajectories that contain a pixel for which error concealment is complete with the respective pixels for which error concealment is complete; and
indicating that error concealment is complete for the replaced pixels.
14 . The method of claim 13 , wherein the performing region filling comprises:
selecting respective INTER-blocks for region filling at least in part by determining whether error concealment is complete for respective pixels in the respective INTER-blocks and blocks that neighbor the respective INTER-blocks; and
replacing pixels in the respective INTER-blocks with pixels obtained from one or more blocks that neighbor the respective INTER-blocks for which error concealment has completed or a weighted average of pixels obtained from two or more blocks that neighbor the respective INTER-blocks for which error concealment has completed.
15 . The method of claim 11 , wherein the performing error concealment comprises:
determining whether respective pixels in the one or more INTER-frames are located within an INTRA-block or an INTER-block;
indicating that error concealment is complete for respective pixels located within INTRA-blocks;
performing motion compensation for pixels located within the respective INTER-blocks;
performing DC coefficient refinement for a first set of selected pixels located within the respective INTER-blocks; and
performing spatial interpolation to refine a second set of selected pixels located within the respective INTER-blocks.
16 . The method of claim 15 , wherein the performing motion compensation comprises:
determining whether respective motion trajectories for pixels located within the respective INTER-blocks contain a pixel for which error concealment is complete;
replacing pixels having motion trajectories that contain a pixel for which error concealment is complete with the respective pixels for which error concealment is complete; and
indicating that error concealment is complete for the replaced pixels.
17 . The method of claim 15 , wherein the performing DC coefficient refinement comprises:
creating the first set of selected pixels at least in part by identifying pixels for which error concealment is not complete that are within a predetermined distance of respective INTRA-blocks; and
replacing respective pixels in the first set of selected pixels with respective weighted averages of original values of the respective pixels and DC coefficients of the respective INTRA-blocks.
18 . The method of claim 15 , wherein the performing spatial interpolation comprises:
creating the second set of selected pixels at least in part by identifying pixels for which error concealment is not complete that are within a predetermined distance of two or more pixels for which error concealment is complete;
generating respective interpolated pixel values based on the two or more pixels for which error concealment is complete that are within a predetermined distance of respective pixels in the second set of selected pixels; and
replacing respective pixels in the second set of selected pixels with respective weighted averages of original values of the respective pixels and the respective interpolated pixel values.
19 . A computer-readable medium having stored thereon instructions operable to perform the method of claim 12 .
20 . A system for processing a video signal, comprising:
means for receiving a video signal on a communication channel, the video signal comprising one or more packets;
means for determining that a packet corresponding to an INTRA-coded frame in the video signal has been lost; and
means for generating video information to replace the lost packet in the video signal and refining subsequent video frames based at least in part on INTRA-coded information provided in one or more correctly received packets.