Image decontouring in high dynamic range video processing
A set of optimized operational parameter values is generated for performing decontouring operations on a predicted image. The predicted image is predicted from a first image mapped from a second image that has a higher dynamic range than the first image. Based on the set of optimized operational parameter values, smoothen operations and selection/masking based on a residual mask are performed on the predicted image. The set of optimized operational parameter values is encoded into a part of a multi-layer video signal that includes the first image, and can be used by a recipient decoder to generate a decontoured image based on the predicted image and reconstruct a version of the second image.
1. A method, comprising:
determining flat image areas in a predicted image, the predicted image being predicted from a first image mapped from a second image that has a higher dynamic range than the first image;
generating, based at least in part on the flat image areas and the predicted image, a set of optimized operational parameter values for decontouring operations;
performing, based at least in part on the set of optimized operational parameter values for decontouring operations, one or more smoothen operations on the predicted image to derive a smoothened image;
applying, based at least in part on the set of optimized operational parameter values for decontouring operations, a residual mask to a residual image between the predicted image and the smoothened image to derive a masked residual image;
encoding the set of optimized operational parameter values into a part of a multi-layer video signal that includes the first image.
2. The method as recited in claim 1 , further comprising encoding the residual mask into the multi-layer video signal.
3. The method as recited in claim 1 , wherein the second image is in a plurality of images that form a scene, and wherein image data derived from the plurality of images is encoded in the multi-layer video signal.
4. The method as recited in claim 1 , wherein the first image is encoded as a part of base-layer (BL) image data in the multi-layer video signal.
5. The method as recited in claim 1 , further comprising:
generating residual values based at least in part on the decontoured image and the second image;
applying non-linear quantization to the residual values to generate enhancement layer (EL) image data; and
encoding the EL image data into the multi-layer video signal.
6. The method as recited in claim 1 , wherein a decontoured image is generated based on the masked residual image and the predicted image.
7. The method as recited in claim 1 , wherein the residual mask comprising binary values, and wherein the residual mask is applied to the residual image between the predicted image and the smoothened image through multiplication operations of the binary values in the residual mask.
8. The method as recited in claim 1 , wherein the smoothen operations comprises one or more of resampling operations, sparse filtering operations, or blur operations.
9. The method as recited in claim 1 , wherein the set of optimized operational parameter values for decontouring operations comprises an optimized value for a scaling factor used in resampling operations.
10. The method as recited in claim 1 , wherein the set of optimized operational parameter values for decontouring operations comprises a smoothness threshold used to generate the residual mask.
11. The method as recited in claim 1 , further comprising:
determining one or more image areas with a high confidence level to have false contouring in an image;
calculating one or more smoothness measures with a plurality of different sets of candidate operational parameter values with respect to the flat image areas; and
selecting, based on the one or more smoothness measures, the set of optimal operational parameter values from the plurality of different sets of candidate operational parameter values.
12. The method as recited in claim 1 , wherein the residual mask covers an entire area of the predicted image.
13. The method as recited in claim 1 , wherein the one or more smoothen operations are performed on pixel values for a specific channel in a plurality of channels of a color space.
14. The method as recited in claim 13 , wherein the plurality of channels comprises one or more of a luminance channel, a chroma channel, a red color channel, a blue color channel, a green color channel, or other primary channels.
15. The method as recited in claim 1 , wherein base layer (BL) image data derived from the second images is compressed by a first encoder into the multi-layer video signal, and wherein enhancement layer (EL) image data derived from the second images is compressed by a second encoder in the multi-layer encoder into the multi-layer video signal.
16. A method, comprising:
extracting a set of optimized operational parameter values from a multi-layer video signal that includes a first image mapped from a second image;
performing, based at least in part on the set of optimized operational parameter values for decontouring operations, one or more smoothen operations on a predicted image to derive a smoothened image, the predicted image being predicted from the first image;
applying a residual mask to a residual image between the predicted image and the smoothened image to derive a masked residual image;
reconstructing a version of the second image based at least in part on the decontoured image.
17. The method as recited in claim 16 , further comprising extracting the residual mask from the multi-layer video signal.
18. The method as recited in claim 16 , wherein the first image is encoded as a part of base-layer (BL) image data in the multi-layer video signal.
19. The method as recited in claim 16 , wherein the method is performed by a video decoder, and wherein the residual mask is constructed by the video decoder based at least in part on the set of optimized operational parameter values for decontouring operations.
20. The method as recited in claim 16 , wherein the residual mask covers an entire area of the predicted image.
21. The method as recited in claim 16 , wherein the one or more smoothen operations are performed on pixel values for a specific channel in a plurality of channels of a color space.
22. The method as recited in claim 21 , wherein the plurality of channels comprises one or more of a luminance channel, a chroma channel, a red color channel, a blue color channel, a green color channel, or other primary channels.
23. A computing device comprising one or more processors and one or more storage media, storing a set of instructions, which when executed by one or more processors cause performance of the method cited in claim 1 .
24. A system performing the method as recited in claim 1 .
25. A non-transitory computer readable storage medium, storing software instructions, which when executed by one or more processors cause performance of the method recited in any of claim 1 .
26. A computing device comprising one or more processors and one or more storage media, storing a set of instructions, which when executed by one or more processors cause performance of the method recited in any of claim 16 .