Video decoding with error detection and concealment
View Patent ↗A system and method of decoding input video information is disclosed which includes performing error detection for each video block of a frame, determining whether a scene change occurs for the frame, and when an error is detected in a video block, performing spatial concealment by concealing error of the erroneous video block using neighboring video information within the frame when the erroneous video block is intraframe encoded or when a scene change is detected for the frame, or performing temporal concealment by replacing the erroneous video block with a reference video block from a reference frame when the erroneous video block is interframe encoded and when a scene change is not detected for the frame. The method may further include detecting false frames based on comparing current and new frame number and picture order count values of a new slice.
1. A method of decoding input video information, comprising:
performing error detection for each video block of a frame;
determining whether a scene change occurs for the frame;
tracking a relative amount of interframe mode video blocks for the frame;
determining error concealment for the frame on a block-by-block basis, comprising:
when an error is detected in a video block of the frame, determining spatial concealment of the erroneous video block when the erroneous video block is intraframe encoded, or when a scene change is detected for the frame and the relative amount of interframe mode video blocks does not achieve a first predetermined threshold, and otherwise determining temporal concealment of the erroneous video block; and
enabling spatial concealment and temporal concealment to be determined within the same frame.
2. The method of claim 1 , wherein said determining whether a scene change occurs for the frame comprises determining whether an amount of intraframe mode video blocks achieves a predetermined percentage of the frame.
3. The method of claim 1 , wherein said determining whether a scene change occurs for the frame comprises determining that a number of intraframe mode video blocks divided by a total number of decoded video blocks achieves a second predetermined threshold.
4. The method of claim 1 , further comprising:
detecting an error of the input video information;
receiving a new frame number from a new video slice; and
decoding the new video slice as part of a current frame when an error of the input video information is detected and when a prior frame number is not zero and is equal to the new frame number.
5. The method of claim 1 , further comprising:
detecting an error of the input video information;
receiving a new frame number and a new picture order count from a new video slice; and
decoding the new video slice as part of a current frame when an error of the input video information is detected and when the prior frame number is not zero and a prior picture order count is equal to the new picture order count.
6. The method of claim 1 , further comprising:
detecting an error of the input video information;
receiving a new frame number from a new video slice; and
decoding the new video slice as part of a current frame when an error of the input video information is detected, when the new frame number is not zero and when a difference between the prior frame number and the new frame number is greater than a predetermined maximum number.
7. The method of claim 1 , further comprising performing spatial concealment comprising interpolating pixel values across the erroneous video block by combining relative portions of first and second aligned pixel values of adjacent video blocks within the frame based on relative position within the erroneous video block.
8. The method of claim 1 , wherein said performing spatial concealment comprises padding pixel values by copying an aligned pixel value of an adjacent video block within the frame across the erroneous video block.
9. The method of claim 1 , further comprising performing spatial concealment comprising:
when an upper adjacent video block and a lower adjacent video block of an erroneous video block are both valid within the frame, interpolating pixel values across the erroneous video block between a lower pixel line of the upper adjacent video block and an upper pixel line of the lower adjacent video block; and
when only one of the upper adjacent video block and the lower adjacent video block is valid within the frame, padding pixel values across the erroneous video block based on an adjacent pixel line of the valid adjacent video block.
10. The method of claim 9 , wherein said performing spatial concealment further comprises:
when the upper adjacent video block and the lower adjacent video block are both invalid and a left adjacent video block and a right adjacent video block of the erroneous video block are both valid, interpolating pixel values across the erroneous video block between a right pixel line of the left adjacent video block and a left pixel line of the right adjacent video block; and
when only one of the right adjacent video block and the left adjacent video block is valid, padding pixel values across the erroneous video block based on an adjacent pixel line of the valid adjacent video block.
11. An error detection and concealment system for a video decoder which is configured to decode each video block of a frame of input video information based on encoding type, said error detection and concealment system comprising:
a scene change detection circuit which is configured to determine a scene change for the frame based on relative amount of intraframe type video blocks;
a tracking circuit which is configured to track a relative amount of interframe type video blocks;
an error detection circuit which is configured to indicate whether there is an error in each video block and
to determine error concealment for the frame on a block-by-block basis, wherein said error concealment circuit is configured to determine spatial concealment for an erroneous video block when said erroneous video block is intraframe encoded, or when a scene change is detected for the frame and said relative amount of interframe mode video blocks does not achieve a first predetermined threshold, and which is configured to otherwise determine temporal concealment of said erroneous video block; and
wherein said error detection circuit is configured to determine spatial concealment and temporal concealment within the same frame.
12. The error detection and concealment system of claim 11 , wherein:
said tracking circuit tracks said relative amount of intraframe type video blocks; and
wherein said scene change detection circuit comprises a comparator circuit which detects a scene change when an amount of intraframe type video blocks achieves a second predetermined threshold.
13. The error detection and concealment system of claim 11 , further comprising a frame error concealment circuit which receives a new frame number and a new picture order count from a new video slice, and which, when an error is detected in the input video information, indicates that said new video slice is to be decoded as part of the frame when a prior frame number is not zero and is equal to said new frame number, when said prior frame number is not zero and a prior picture order count is equal to said new picture order count, or when said new frame number is not zero and when a difference between said prior frame number and said new frame number is greater than a predetermined maximum number.
14. The error detection and concealment system of claim 11 , further comprising an error concealment circuit which performs said spatial concealment by either interpolating pixel values between valid adjacent video blocks on either side of said erroneous video block or by padding pixel values from one valid adjacent video block.
15. The error detection and concealment system of claim 14 , wherein said error concealment circuit performs interpolation by combining relative portions of first and second aligned pixel values based on relative position within said erroneous video block, and wherein said error concealment circuit performs padding by copying an aligned pixel value across said erroneous video block.
16. A video decoder, comprising:
a video decoder circuit which is configured to decode video blocks of a frame based on encoding type;
a scene change detection circuit which is configured to determine whether a scene change is occurring for said frame based on relative amount of intraframe encoded video blocks;
an error memory;
a tracking circuit which is configured to track a relative amount of interframe encoded video blocks;
an error detection circuit which is configured to determine error concealment type for said frame on a block-by-block basis and to program said error memory to indicate whether there is an error in each video block and a type of error concealment between spatial error concealment and temporal error concealment based on video block encoding type and whether a scene change is occurring; and
an error concealment circuit which is configured to perform spatial concealment for an erroneous video block by incorporating neighboring video information within said frame when said erroneous video block is intraframe encoded, or when a scene change is detected for the frame and said relative amount of interframe encoded video blocks does not achieve a predetermined threshold, and which otherwise is configured to perform temporal concealment by replacing said erroneous video block with a corresponding video block from a reference frame;
wherein said error concealment circuit is configured to perform both spatial concealment and temporal concealment within said frame.
17. The video decoder of claim 16 , further comprising a frame error concealment circuit which receives a new frame number and a new picture order count from a new video slice, and which, after detecting an error of input video information, causes said video decoder circuit to decode said new video slice as part of the frame when a prior frame number is not zero and is equal to said new frame number, when said prior frame number is not zero and a prior picture order count is equal to said new picture order count, or when said new frame number is not zero and when a difference between said prior frame number and said new frame number is greater than a predetermined maximum number.
18. The method of claim 1 , while decoding a frame of the input video information, further comprising:
counting a number of video blocks received for decoding so far in the frame and providing a video block count;
counting a number of intraframe mode video blocks received for decoding so far in the frame and providing an intraframe count;
counting a number of interframe mode video blocks received for decoding so far in the frame and providing an interframe count;
determining that an erroneous video block is a lost video block when its coding mode is unknown; and
when the video block count achieves a predetermined percentage of the frame, providing a scene change indication when the intraframe count divided by the video block count achieves a first predetermined threshold, and providing an intermode indication when the interframe count divided by the video block count achieves a second predetermined threshold;
said determining spatial concealment comprising determining spatial concealment of the erroneous video block when the erroneous video block is intraframe encoded, or when the scene change indication is provided and the erroneous video block is a lost video block, or when the erroneous video block is interframe encoded when the scene change indication is provided and the intermode indication is not provided; and
said determining temporal concealment comprising determining temporal concealment of the erroneous video block when the erroneous video block is not intraframe encoded and the scene change indication is not provided, or when the erroneous video block is interframe encoded and the scene change indication and the interframe indication are both provided.
19. The error detection and concealment system of claim 11 , wherein:
said tracking circuit is configured to count a number of video blocks received for decoding so far in the frame and to provide a video block count indicative thereof, to count a number of intraframe mode video blocks received for decoding so far in the frame and to provide an intraframe count indicative thereof, to count a number of interframe mode video blocks received for decoding so far in the frame and to provide an interframe count indicative thereof, and to determine whether a video block is a lost video block when its coding mode is unknown;
wherein said scene change detection circuit comprises a comparator circuit which is configured to provide a scene change indication when said intraframe count divided by said video block count achieves a first predetermined threshold after said video block count has achieved a predetermined percentage of the frame, and to provide an intermode indication when said interframe count divided by said video block count achieves a second predetermined threshold after said video block count has achieved a predetermined percentage of the frame;
wherein said error detection circuit is configured to determine spatial concealment of said erroneous video block when said erroneous video block is intraframe encoded, or when said erroneous video block is a lost video block and said scene change indication is provided, or when said erroneous video block is interframe encoded when said scene change indication is provided and said intermode indication is not provided; and
wherein said error concealment circuit is further configured to determine temporal concealment of said erroneous video block when said erroneous video block is not intraframe encoded and said scene change indication is not provided, or when said erroneous video block is interframe encoded and said scene change indication and said interframe indication are both provided.
20. The video decoder of claim 16 , wherein:
said tracking circuit is configured to count a number of video blocks received for decoding so far in the frame and provides a video block count, to count a number of intraframe mode video blocks received for decoding so far in the frame and provides a corresponding intraframe count, to count a number of interframe mode video blocks received for decoding so far in the frame and provides a corresponding interframe count, and to determine whether a video block is a lost video block when its coding mode is unknown;
wherein said scene change detection circuit comprises a comparator circuit which is configured to provide a scene change indication when said intraframe count divided by said video block count achieves a first predetermined threshold after said video block count has achieved a predetermined percentage of the frame, and to provide an intermode indication when said interframe count divided by said video block count achieves a second predetermined threshold after said video block count has achieved a predetermined percentage of the frame;
wherein said error concealment circuit is configured to perform spatial concealment of said erroneous video block when said erroneous video block is intraframe encoded, or when said erroneous video block is a lost video block and said scene change indication is provided, or when said erroneous video block is interframe encoded when said scene change indication is provided and said intermode indication is not provided; and
wherein said error concealment circuit is further configured to perform temporal concealment of said erroneous video block when said erroneous video block is not intraframe encoded and said scene change indication is not provided, or when said erroneous video block is interframe encoded and said scene change indication and said interframe indication are both provided.