Systems and methods for enhanced error concealment in a video decoder
View Patent ↗The invention is related to methods and apparatus that conceal errors in images of a corrupted video bitstream. One embodiment conceals errors in a missing or corrupted intra-coded macroblock by linearly interpolating data from other macroblocks that correspond to portions of the image above and below the missing or corrupted macroblock. One embodiment can utilize substitute motion vectors for a missing or corrupted predictive-coded macroblock. Another embodiment doubles the received motion vectors and references the doubled motion vectors to a previous-previous frame. Another embodiment adaptively selects which concealment or reconstruction technique is applied according to projected error estimates. Another embodiment conceals errors by replacing corrupted or missing data by combining concealment data in a weighted sum to reduce an estimated error.
1. A method of adaptively producing a video image comprising:
receiving video data for a frame; determining whether the video data is intra-coded or predictive-coded; when the video data is intra-coded: determining whether the intra-coded video data corresponds to an error; concealing the error when the intra-coded video data corresponds to the error; setting an error value that is associated with at least a portion of the video packet to a first predetermined value when the intra-coded video data corresponds to the error; resetting the error value when no error for the intra-coded video data is detected; and using the intra-coded video data when no error for the intra-coded video data is detected; when the video data is predictive-coded, determining whether the predictive-coded video data corresponds to an error; when the predictive-coded video data corresponds to an error: using the predictive-coded video data when no error for the predictive-coded video data is detected and the associated error value is reset; projecting a first estimated error corresponding to use of the predictive-coded video data when no error is detected for the predictive-coded video data and the associated error value is not reset; projecting a second estimated error corresponding to use of a first predictive-coded error concealment technique when no error is detected for the predictive-coded video data and the associated error value is not reset; selecting between the use of the predictive-coded video data and the use of the first predictive-coded error concealment technique based on a comparison between the first projected estimated error and the second projected estimated error; and updating the error value according to which of the predictive-coded video data and the first predictive-coded error concealment technique is selected; and when the predictive-coded video data corresponds to an error: applying a second predictive-coded error concealment technique; and updating the error value according to the second predictive-coded error concealment technique.
2. The method as defined in claim 1 , wherein the first predictive-coded error concealment technique and the second predictive-coded error concealment technique are the same.
3. The method as defined in claim 1 , wherein the projecting a second estimated error further comprises projecting a plurality of estimated errors corresponding to a plurality of error concealment techniques for predictive coding, and wherein the selecting between the use of the predictive-coded video data and the use of the predictive-coded error concealment technique further comprises selecting among the use of the predictive-coded video data and the use of an error concealment technique from the plurality of error concealment techniques based on the corresponding estimated error projections.
4. The method as defined in claim 1 , wherein the applying the second predictive-coded error concealment technique further comprises:
projecting a plurality of estimated errors corresponding to a plurality of error concealment techniques for predictive coding; using the projected estimate errors to select among the plurality of error concealment techniques; applying the selected error concealment technique; and adjusting the error value according to the selected error concealment technique.
5. The method as defined in claim 1 , wherein the video data is a macroblock.
6. The method as defined in claim 1 , wherein the video data is a video object plane (VOP).
7. The method as defined in claim 1 , wherein the video data is a frame.
8. The method as defined in claim 1 , further comprising normalizing the error value to a range between 0 to 255.
9. The method as defined in claim 1 , further comprising multiplying the error value with a leaky value that has a value of less than 1 in response to an advancement in a frame sequence.
10. The method as defined in claim 9 , wherein the leaky value is about 0.93.
11. The method as defined in claim 1 , further comprising maintaining the error value in a memory array, wherein an error value in the array is associated with at least one pixel in the image.
12. The method as defined in claim 1 , further comprising maintaining the error value in a memory array, wherein each pixel in the image is associated with an error value in the array.
13. A method of producing a video image comprising: receiving data for a video frame; determining whether the video frame is a predictive-coded frame or is an intra-coded frame; performing the following when the video frame is the predictive-coded frame: determining whether a group of video data from the video frame corresponds to an error; when there is no error in the group of video data: determining whether the group of video data is intra-coded or predictive-coded; intra-decoding the group of video data when the group of video data is intra coded; resetting an error variance associated with at least a portion of the group of video data when the group of video data is intra coded; using a first weighted sum to reconstruct a portion of an image corresponding to the group of video data when the video data is intra coded, where the first weighted sum combines results of at least a first and a second technique; and updating the error variance according to the first weighted sum used to reconstruct the portion of the image; and when there is an error in the group of video data: concealing the error in the portion of the image corresponding to the group of video data; and updating the error variance according to the error concealment.
14. The method as defined in claim 13 , wherein the group of video data comprises a macroblock.
15. The method as defined in claim 13 , wherein the group of video data comprises a video object plane (VOP).
16. The method as defined in claim 13 , wherein the group of video data comprises missing data.
17. The method as defined in claim 13 , wherein the concealing the error further comprises using a second weighted sum to conceal the portion of the image corresponding to the group of video data, where the second weighted sum combines results of at least at least two error concealing techniques.
18. The method as defined in claim 13 , wherein the first weighted sum weighs the results of the first and the second technique according to values that are related to inverses of expected errors of the first and the second techniques.
19. The method as defined in claim 13 , wherein the first technique comprises constructing the portion of the image from a first reference portion of a previous frame and the second technique comprises constructing the portion of the image from a second reference portion of a frame that is prior to the previous frame.
20. The method as defined in claim 13 , wherein the second weighted sum weighs the results of the third and the fourth error concealing techniques according to inverses of expected errors of the third and the fourth error concealing techniques, respectively.
21. The method as defined in claim 13 , when the video frame is the predictive-coded frame, further comprising: receiving a next group of video data; and continuing execution of the method until the groups of video data are processed.
22. The method as defined in claim 13 , further comprising: performing the following when the video frame is the intra-coded frame: determining whether a group of data from the corresponds to an error; when there is no error in the group of video data: intra-decoding the group of video data; and resetting an error variance associated with at least a portion of the group of video data; and when there is error in the group of video data: concealing the error in the portion of the image corresponding to the group of video data; and setting the error variance to a predetermined value.
23. The method as defined in claim 22 , when the video frame is the intra-coded frame, further comprising: receiving a next group of video data; and continuing execution of portions of the method corresponding to groups of data in an intra-decoded frame until the groups of video data are processed.