Sign prediction for block-based video coding

1 . A video decoding method, comprising:

receiving a bitstream comprising a sequence of sign signaling bits;

determining a sign prediction area in a transform block of a video frame for performing sign prediction for transform coefficients of the transform block;

generating a plurality of candidate hypotheses for a set of candidate transform coefficients associated with the sign prediction area of the transform block;

obtaining a histogram of gradient (HoG) based on an accumulated magnitude of gradients for each of a plurality of angular directions;

when a ratio of a magnitude of a dominant gradient direction with the largest magnitude in the HoG to a sum of magnitudes of all gradient directions in the HoG is larger than a predetermined threshold, calculating a cost function by extrapolating neighboring samples of the transform block in an extrapolation direction determined based on the dominant gradient direction;

selecting a hypothesis from the plurality of candidate hypotheses as a set of predicted signs for the set of candidate transform coefficients based on the cost function; and

estimating original signs for the set of candidate transform coefficients based on the set of predicted signs and the sequence of sign signaling bits.

2 . The video decoding method of claim 1 , wherein the sign prediction area is determined based on a size of the transform block and an area size threshold.

3 . The video decoding method of claim 2 , wherein a width of the sign prediction area is determined to be the smaller of a width of the transform block and the area size threshold, and a height of the sign prediction area is determined to be the smaller of a height of the transform block and the area size threshold.

4 . The video decoding method of claim 2 , wherein the area size threshold is a fixed value for all transform blocks of the video, wherein the area size threshold is not signaled in the bitstream; or

wherein the area size threshold is a value adaptively determined for the transform block, wherein the value of the area size threshold is signaled in the bitstream; or wherein the value of the area size threshold is selected from a group of predetermined values, wherein the value of the area size threshold is signaled using a fixed-length codeword; or

wherein the value of the area size threshold is an arbitrary value determined for the transform block, wherein the value of the area size threshold is signaled using a variable-length codeword; or

wherein the value of the area size threshold is signaled in a video parameter set, a sequence parameter set, a picture parameter set, or a slice header and coding block level.

5 . The video decoding method of claim 1 , wherein the bitstream further comprises quantized transform coefficients associated with the transform block, wherein the video decoding method further comprises:

generating dequantized transform coefficients from the quantized transform coefficients; and

updating the dequantized transform coefficients based on the estimated original signs for the set of candidate transform coefficients.

6 . The video decoding method of claim 1 , further comprising:

applying an inverse primary transform and an inverse low-frequency non-separable transform (LFNST) to dequantized transform coefficients associated with the sign prediction area to generate residual samples in a residual block corresponding to the transform block.

7 . The video decoding method of claim 1 , wherein generating the plurality of candidate hypotheses for the set of candidate transform coefficients further comprises:

determining a plurality of combinations of sign candidates for the set of candidate transform coefficients based on a total number of candidate transform coefficients in the set of candidate transform coefficients; and

applying a template-based hypothesis generation scheme to generate the plurality of candidate hypotheses for the plurality of combinations of sign candidates, respectively.

8 . The video decoding method of claim 1 , wherein the sequence of sign signaling bits for the set of candidate transform coefficients indicate correctness of the predicted signs of the respective candidate transform coefficients.

9 . The video decoding method of claim 8 , wherein estimating the original signs for the set of candidate transform coefficients based on the set of predicted signs and the sequence of sign signaling bits comprises:

adopting or correcting the predicted signs of the candidate transform coefficients according to the correctness indicated by the respective sign signaling bits to determine the original signs of the candidate transform coefficients.

10 . The video decoding method of claim 8 , wherein the sequence of sign signaling bits are entropy-coded based on a context of magnitudes of the respective candidate transform coefficients; or

wherein the sequence of sign signaling bits are entropy-coded based on a context of scan positions of the respective candidate transform coefficients; or

wherein the sequence of sign signaling bits are entropy-coded based on a context of a coding mode, a block size or component channel information of the transform block.

11 . A video decoding apparatus, comprising:

a memory configured to store a bitstream comprising a sequence of sign signaling bits; and

one or more processors coupled to the memory and configured to:

determine a sign prediction area in a transform block of a video frame for performing sign prediction for transform coefficients of the transform block;

generate a plurality of candidate hypotheses for a set of candidate transform coefficients associated with the sign prediction area of the transform block;

obtain a histogram of gradient (HoG) based on an accumulated magnitude of gradients for each of a plurality of angular directions;

when a ratio of a magnitude of a dominant gradient direction with the largest magnitude in the HoG to a sum of magnitudes of all gradient directions in the HoG is larger than a predetermined threshold, calculate a cost function by extrapolating neighboring samples of the transform block in an extrapolation direction determined based on the dominant gradient direction;

select a hypothesis from the plurality of candidate hypotheses as a set of predicted signs for the set of candidate transform coefficients based on the cost function; and

estimate original signs for the set of candidate transform coefficients based on the set of predicted signs and the sequence of sign signaling bits.

12 . The video decoding apparatus of claim 11 , wherein the sign prediction area is determined based on a size of the transform block and an area size threshold.

13 . The video decoding apparatus of claim 12 , wherein a width of the sign prediction area is determined to be the smaller of a width of the transform block and the area size threshold, and a height of the sign prediction area is determined to be the smaller of a height of the transform block and the area size threshold.

14 . The video decoding apparatus of claim 12 , wherein the area size threshold is a fixed value for all transform blocks of the video, wherein the area size threshold is not signaled in the bitstream; or

wherein the area size threshold is a value adaptively determined for the transform block, wherein the value of the area size threshold is signaled in the bitstream; or

wherein the value of the area size threshold is selected from a group of predetermined values, wherein the value of the area size threshold is signaled using a fixed-length codeword; or

wherein the value of the area size threshold is an arbitrary value determined for the transform block, wherein the value of the area size threshold is signaled using a variable-length codeword; or

wherein the value of the area size threshold is signaled in a video parameter set, a sequence parameter set, a picture parameter set, or a slice header and coding block level.

15 . The video decoding apparatus of claim 11 , wherein the sequence of sign signaling bits for the set of candidate transform coefficients indicate correctness of the predicted signs of the respective candidate transform coefficients.

16 . The video decoding apparatus of claim 15 , wherein to estimate the original signs for the set of candidate transform coefficients based on the set of predicted signs and the sequence of sign signaling bits, the one or more processors are configured to:

adopt or correct the predicted signs of the candidate transform coefficients according to the correctness indicated by the respective sign signaling bits to determine the original signs of the candidate transform coefficients.

17 . The video decoding apparatus of claim 15 , wherein the sequence of sign signaling bits are entropy-coded based on a context of magnitudes of the respective candidate transform coefficients; or

wherein the sequence of sign signaling bits are entropy-coded based on a context of scan positions of the respective candidate transform coefficients; or

wherein the sequence of sign signaling bits are entropy-coded based a context of on a coding mode, a block size or component channel information of the transform block.

18 . A non-transitory computer-readable storage medium having stored therein a bitstream, which is decodable by a video decoding method according to claim 1 .

19 . A video encoding method, comprising:

determining, a sign prediction area in a transform block of a video frame for performing the sign prediction for transform coefficients of the transform block;

generating, a plurality of candidate hypotheses for a set of candidate transform coefficients associated with the sign prediction area of the transform block;

obtaining a histogram of gradient (HoG) based on an accumulated magnitude of gradients for each of a plurality of angular directions;

when a ratio of a magnitude of a dominant gradient direction with the largest magnitude in the HoG to a sum of magnitudes of all gradient directions in the HoG is larger than a predetermined threshold, calculating a cost function by extrapolating neighboring samples of the transform block in an extrapolation direction determined based on the dominant gradient direction;

selecting, a hypothesis from the plurality of candidate hypotheses as a set of predicted signs for the set of candidate transform coefficients based on the cost function;

estimating original signs for the set of candidate transform coefficients based on the set of predicted signs and a sequence of sign signaling bits; and

signaling a bitstream comprising the sequence of sign signaling bits.

20 . A method of storing a bitstream comprising:

performing a video encoding method according to claim 19 to generate a bitstream comprising a sequence of sign signaling bits; and

storing the bitstream.