Content adaptive deblocking during video encoding and decoding
Disclosed herein are exemplary embodiments of methods, apparatus, and systems for performing content-adaptive deblocking to improve the visual quality of video images compressed using block-based motion-predictive video coding. For instance, in certain embodiments of the disclosed technology, edge information is obtained using global orientation energy edge detection (“OEED”) techniques on an initially deblocked image. OEED detection can provide a robust partition of local directional features (“LDFs”). For a local directional feature detected in the partition, a directional deblocking filter having an orientation corresponding to the orientation of the LDF can be used. The selected filter can have a filter orientation and activation thresholds that better preserve image details while reducing blocking artifacts. In certain embodiments, for a consecutive non-LDF region, extra smoothing can be imposed to suppress the visually severe blocking artifacts.
1. A method for reducing block artifacts during video compression or decompression, comprising:
buffering a video frame reconstructed during block-based motion-predictive encoding or decoding;
determining edge locations and edge orientations throughout one or more blocks in the video frame;
for a selected block of the one or more blocks in the video frame, selecting a deblocking filter from two or more candidate deblocking filters based at least in part on the edge orientations in the selected block, the candidate deblocking filters including two or more directional deblocking filters adapted to reduce blockiness at a block boundary while also maintaining a real edge of a directional feature of the video frame at the block boundary, each of the directional deblocking filters being adapted for a different non-horizontal and non-vertical orientation of the directional feature of the video frame, each of the directional deblocking filters further comprising a filter bank of multiple filters in which the filters in the filter bank have the same non-horizontal and non-vertical orientation as one another; and
applying the selected deblocking filter to the selected block and one or more blocks neighboring the selected block.
2. The method of claim 1 , wherein the determining the edge locations and the edge orientations comprises partitioning the blocks of the video frame into blocks having local directional features and blocks having no local directional features using orientation energy edge detection.
3. The method of claim 2 , wherein the determining the edge locations and the edge orientations comprises applying two or more orientation energy filters, at least one of the orientation energy filters being in a non-vertical and non-horizontal orientation.
4. The method of claim 1 , wherein the determining the edge locations and the edge orientations comprises applying sets of one or more Gaussian derivative filters, each set of the one or more Gaussian derivative filters having Gaussian derivative filters at an orientation different from the Gaussian derivative filters of the other sets.
5. The method of claim 4 , wherein the sets of one or more Gaussian derivative filters comprise a first Gaussian derivative filter and a second Gaussian derivative filter.
6. The method of claim 4 , wherein eight sets of one or more Gaussian derivative filters are applied, six of the eight sets being at a non-horizontal and non-vertical orientation.
7. The method of claim 1 , further comprising applying a deblocking filter to luminance values of the video image before the edge locations and the edge orientations are determined.
8. The method of claim 1 , wherein the method is performed in a motion estimation/compensation loop of an encoder or in a motion compensation loop of a decoder.
9. The method of claim 1 , wherein the act of selecting a deblocking filter comprises determining whether the selected block has directional features that are consistent with directional features of the one or more blocks neighboring the selected block.
10. The method of claim 9 , wherein the determination of whether the selected block has directional features that are consistent with the directional features of the one or more blocks neighboring the selected block is based at least in part on one or more of an average orientation of edge pixels and an orientation variance of the edge pixels.
11. A method for reducing block artifacts during video encoding or decoding, comprising:
buffering a video frame reconstructed during block-based motion-predictive encoding or decoding;
for a selected block in the video frame, determining whether the selected block is in a region of two or more blocks with no local directional features, wherein the determining is based on intensity variations of pixels within the selected block as well as intensity variations of pixels within one or more adjacent blocks;
selecting one of a plurality of available filters to apply to the selected block based at least in part on the determination of whether the selected block is in a region with no local directional features, the available filters including a smoothing filter and a plurality of directional deblocking filters, each of the directional deblocking filters further comprising a filter bank of multiple filters in which the filters in the filter bank have the same non-horizontal and non-vertical orientation as one another; and
applying the selected filter to the selected block.
12. The method of claim 11 , wherein the region with no local directional features comprises the selected block and two or more adjacent blocks to the selected block.
13. The method of claim 11 , wherein the selecting comprises selecting the smoothing filter when the selected block is determined to be in a region with no local directional features.
14. The method of claim 11 , wherein the determining whether the selected block is in a region with no local directional features comprises determining whether a sum of the intensity variations in the selected block and in the one or more adjacent blocks satisfy a threshold value.
15. The method of claim 11 , wherein the determining whether the selected block is in a region with no local directional features further comprises partitioning the decoded image into blocks having local directional features and blocks having no local directional features using an edge detection filter and determining that the selected block has no local directional features based at least in part on the partitioning.
16. The method of claim 11 , wherein the method is performed in a motion estimation/compensation loop of an encoder or in a motion estimation loop of a decoder.
17. One or more computer memory devices or one or more computer-readable storage devices storing computer-executable instructions which when executed by a computer cause the computer to perform a video encoding or video decoding method, the method comprising:
detecting edge locations and edge orientations in a video image;
for a block of the video image having a detected edge, selecting a directional deblocking filter from among multiple directional deblocking filters, each of the directional deblocking filters being adapted for different non-horizontal and non-vertical directional feature orientations and being adapted to reduce blockiness at a boundary of the block while also maintaining an edge and orientation of a directional feature at the boundary of the block, each of the directional deblocking filters further comprising a filter bank of multiple filters in which the filters in the filter bank have the same non-horizontal and non-vertical orientation as one another;
evaluating one or more application thresholds for the selected directional deblocking filter;
if the application thresholds for the directional deblocking filter are satisfied, applying the directional deblocking filter across the boundary of the block and at least one adjacent block; and
if the application thresholds for the directional deblocking filter are not satisfied, applying no deblocking filter to the block.
18. The one or more computer memory devices or one or more computer-readable storage devices of claim 17 , wherein the edge locations and edge orientations are detected by applying two or more orientation energy filters to the video image.
19. The one or more computer memory devices or one or more computer-readable storage devices of claim 17 , wherein at least one of the application thresholds for the selected directional deblocking filter is dependent on the orientation of the selected directional deblocking filter.
20. The one or more computer memory devices or one or more computer-readable storage devices of claim 17 , wherein the block is a first block, and wherein the method further comprises:
for a second block of the video image having no detected edge, determining that the second block has one or more adjacent blocks that also have no detected edge; and
applying a smoothing filter to pixels in the second block and the one or more adjacent blocks.