Gradient based boundary filtering in intra prediction
After initial intra prediction based on projection along an intra prediction direction, there may exist discontinuities between the predicted block and neighboring blocks. To smooth the block boundary, boundary filtering may be used with intra prediction. In particular, different methods may be used for boundary filtering. The particular filtering method to be used may be selected adaptively based on the intra prediction mode. When gradient-based filtering is selected, the gradient at a reference sample, that is at the same row or column as the current sample, in a reference layer is calculated as the difference between the reference sample and the predicted value for the reference sample using the intra prediction mode. Additional gradient at another reference sample from another reference layer may also be used. Then the initially predicted value for the sample may be adjusted based on the gradient(s) and a decay function.
1. A method for video decoding, comprising:
accessing a predicted value of a sample of a block of an image in a video, corresponding to a vertical intra prediction mode;
determining, based on a location of said sample in said block and said vertical intra prediction mode, that said predicted value of said sample is to be adjusted;
adjusting said predicted value of said sample, responsive to a difference of a reference sample and a predicted value of said reference sample using said vertical intra prediction mode and a scaling factor, wherein said reference sample is from a left reference array and is at a same vertical position as said sample, wherein said scaling factor depends on said horizontal location of said sample in said block and a size of said block, said adjusting including:
multiplying said difference of said reference sample and said predicted value of said reference sample by said scaling factor to obtain a scaled difference,
adding said scaled difference to an integer N to form a sum, and
right shifting said sum by M bits to obtain an adjusted predicted value of said sample, wherein 2 M-1 =N;
accessing a prediction residual of said sample; and
decoding said sample responsive to a sum of said adjusted predicted value and said prediction residual.
2. The method of claim 1 , wherein said predicted value for said reference sample is a top-left reference sample for said block.
3. The method of claim 2 , wherein said scaling factor=32>>((x<<1)>>s), s=(log 2(W)−1+log 2(H)−1)>>2, and W and H are width and height of said block, respectively.
4. The method of claim 1 , wherein said sample is determined to be adjusted when said sample is close to a left boundary of said block and said vertical intra prediction mode corresponds to a directly vertical prediction direction.
5. The method of claim 1 , wherein M=6 and N=32.
6. A method for video encoding, comprising:
accessing a predicted value of a sample of a block of an image in a video, corresponding to a vertical intra prediction mode;
determining, based on a location of said sample in said block and said vertical intra prediction mode, that said predicted value of said sample is to be adjusted;
adjusting predicted value of said sample, responsive to a difference of reference sample and a predicted value of said reference sample using said vertical intra prediction mode and a scaling factor, wherein said reference sample is from a left reference array and is at a same vertical position as said sample, wherein said scaling factor depends on said horizontal location of said sample in said block and a size of said block, said adjusting including:
multiplying said difference of said reference sample and said predicted value of said reference sample by said scaling factor to obtain a scaled difference,
adding said scaled difference to an integer N to form a sum, and
right shifting said sum by M bits to obtain an adjusted predicted value of said sample, wherein 2 M-1 =N;
generating a prediction residual for said sample responsive to a difference of said sample and said adjusted predicted value; and
encoding said vertical intra prediction mode and said prediction residual.
7. The method of claim 6 , wherein said predicted value for said reference sample is a top-left reference sample for said block.
8. The method of claim 7 , wherein said scaling factor=32>>((x<<1)>>s), s=(log 2(W)−1+log 2(H)−1)>>2, and W and H are width and height of said block, respectively.
9. The method of claim 6 , wherein said sample is determined to be adjusted when said sample is close to a left boundary of said block and said vertical intra prediction mode corresponds to a directly vertical prediction direction.
10. The method of claim 6 , wherein M=6 and N=32.
11. An apparatus for decoding video data, comprising:
one or more processors and at least a memory, wherein said one or more processors are configured to:
access a predicted value of a sample of a block of an image in a video, corresponding to a vertical intra prediction mode;
determine, based on a location of said sample in said block and said vertical intra prediction mode, that said predicted value of said sample is to be adjusted;
adjust said predicted value of said sample, responsive to a difference of a reference sample and a predicted value of said reference sample using said vertical intra prediction mode and a scaling factor, wherein said reference sample is from a left reference array and is at a same vertical position as said sample, wherein said scaling factor depends on said location of said sample and a size of said block, wherein said one or more processors are configured to adjust said predicted value of said sample by performing:
multiplying said difference of said reference sample and said predicted value of said reference sample by said scaling factor to obtain a scaled difference,
adding said scaled difference to an integer N to form a sum, and
right shifting said sum by M bits to obtain an adjusted predicted value of said sample, wherein 2 M-1 =N;
access a prediction residual of said sample; and
decode said sample responsive to a sum of said adjusted predicted value and said prediction residual.
12. The apparatus of claim 11 , wherein said predicted value for said reference sample is a top-left reference sample for said block.
13. The apparatus of claim 12 , wherein said scaling factor=32>>((x<<1)>>s), s=(log 2(W)−1+log 2(H)−1)>>2, and W and H are width and height of said block, respectively.
14. The apparatus of claim 11 , wherein said sample is determined to be adjusted when said sample is close to a left boundary of said block and said vertical intra prediction mode corresponds to a directly vertical prediction direction.
15. The apparatus of claim 11 , wherein M=6 and N=32.
16. An apparatus for encoding video data, comprising:
one or more processors and at least a memory, wherein said one or more processors are configured to:
access a predicted value of a sample of a block of an image in a video, corresponding to a vertical intra prediction mode;
determine, based on a location of said sample in said block and said intra prediction mode, that said predicted value of said sample is to be adjusted;
adjust said predicted value of said sample, responsive to a difference of a reference sample and a predicted value of said reference sample using said vertical intra prediction mode and a scaling factor, wherein said reference sample is from a left reference array and is at a same vertical position as said sample, wherein said scaling factor depends on said location of said sample and a size of said block, wherein said one or more processors are configured to adjust said predicted value of said sample by performing:
multiplying said difference of said reference sample and said predicted value of said reference sample by said scaling factor to obtain a scaled difference,
adding said scaled difference to an integer N to form a sum, and
right shifting said sum by M bits to obtain an adjusted predicted value of said sample, wherein 2 M-1 =N;
generate a prediction residual for said sample responsive to a difference of said sample and said adjusted predicted value; and
encode said vertical intra prediction mode and said prediction residual.
17. The apparatus of claim 16 , wherein said predicted value for said reference sample is a top-left reference sample for said block.
18. The apparatus of claim 17 , wherein said scaling factor=32>>((x<<1)>>s), s=(log 2(W)−1+log 2(H)−1)>>2, and W and H are width and height of said block, respectively.
19. The apparatus of claim 16 , wherein said sample is determined to be adjusted when said sample is close to a left boundary of said block and said vertical intra prediction mode corresponds to a directly vertical prediction direction.
20. The apparatus of claim 16 , wherein M=6 and N=32.