Encoder, a decoder and corresponding methods of filter modification on general intra prediction process

1. A method for intra prediction, comprising:

during an intra predicted sample generation stage, obtaining a filtering indication indicating whether a filtered value or an unfiltered value of reference samples of a current coding block is to be used;

in response to determining that a value of the filtering indication is true, generating a prediction sample by using the filtered value of the reference samples based on a planar mode; or

in response to determining that the value of the filtering indication is false, generating a prediction sample by using the unfiltered value of the reference samples based on the planar mode; and

during the intra predicted sample generation stage, obtaining the filtered value of the reference samples by applying a filter to the reference samples, wherein:

when the filter is [1 2 1],

predV[x][y]=((nH−1−y)*((p[x−1][−1]+2*p[x][−1]+p[x+1][−1]+2)>>2)+(y+1) * p[−1][nTbH])<<Log2 (nW),

predH[x][y]=((nW−1−x)*((p[−1][y−1]+2*p[−1][y]+p[−1][y+1]+2)>>2)+(x+1)*p[nTbW][−1])<<Log2 (nH),

wherein predV[x][y] is a vertical component of the prediction sample, predH[x][y] is a horizontal component of the prediction sample, and p[x][y] (x=−1, y=−1 . . . nTbH and x=0 . . . nTbW, y=−1) is the reference sample; and

when the filter is [1 6 1],

predV[x][y]=((nH−1−y)*((p[x−1][−1]+(p[x][−1]*6)+p[x+1][−1]+4)>>3)+(y+1)*p[−1][nTbH])<<Log2 (nW),

predH[x][y]=((nW−1−x)*((p[−1][y−1]+(p[−1][y]*6)+p[−1][y+1]+4)>>3)+(x+1)*p[nTbW][−1])<<Log2 (nH),

wherein predV[x][y] is a vertical component of the prediction sample, predH[x][y] is a horizontal component of the prediction sample, and p[x][y] (x=−1, y=−1 . . . nTbH and x=0 . . . nTbW, y=−1) is the reference sample.

2. The method of claim 1 , wherein the value of the filtering indication is true when one or more of the following conditions are true:

current block size is larger than N luma samples (N could be a number equals to power of 2 and larger than 2, i.e 4, 8, 16, 32, 64, 128, . . . );

current block is NOT applied with intra sub-partition (ISP); or

current block is a luma block.

3. The method of claim 1 , the method further comprises a position-dependent intra prediction (PDPC) stage, during the PDPC stage, reference samples (p[x][y], with x=−1, y=−1 . . . nTbH and x=0 . . . nTbW, y=−1) for the PDPC inputs are not filtered.

4. A decoder comprising:

one or more processors; and

a non-transitory computer-readable storage medium coupled to the one or more processors and storing programming for execution by the one or more processors, wherein the programming, when executed by the processors, configures the decoder to perform a method comprising:

during an intra predicted sample generation stage, obtaining a filtering indication indicating whether a filtered value or an unfiltered value of reference samples of a current coding block is to be used;

in response to determining that a value of the filtering indication is true, generating a prediction sample by using the filtered value of the reference samples based on a planar mode; or

in response to determining that the value of the filtering indication is false, generating a prediction sample by using the unfiltered value of the reference samples based on the planar mode; and

during the intra predicted sample generation stage, obtaining the filtered value of the reference samples by applying a filter to the reference samples, wherein:

when the filter is [1 2 1],

predV[x][y]=((nH−1−y)*((p[x−1][−1]+2*p[x][−1]+p[x+1][−1]+2)>>2)+(y+1)*p[−1][nTbH])<<Log2 (nW),

predH[x][y]=((nW−1−x)*((p[−1][y−1]+2*p[−1][y]+p[−1][y+1]+2)>>2)+(x+1)*p[nTbW][−1])<<Log2 (nH),

wherein predV[x][y] is a vertical component of the prediction sample, predH[x][y] is a horizontal component of the prediction sample, and p[x][y] (x=−1, y=−1 . . . nTbH and x=0 . . . nTbW, y=−1) is the reference sample; and

when the filter is [1 6 1],

predV[x][y]=((nH−1−y)*((p[x−1][−1]+(p[x][−1]*6)+p[x+1][−1]+4)>>3)+(y+1)*p[−1][nTbH])<<Log2 (nW),

predH[x][y]=((nW−1−x)*((p[−1][y−1]+(p[−1][y]*6)+p[−1][y+1]+4)>>3)+(x+1)*p[nTbW][−1])<<Log2 (nH),

wherein predV[x][y] is a vertical component of the prediction sample, predH[x][y] is a horizontal component of the prediction sample, and p[x][y] (x=−1, y=−1 . . . nTbH and x=0 . . . nTbW, y=−1) is the reference sample.

5. The decoder claim 4 , wherein the value of the filtering indication is true when one or more of the following conditions are true:

current block size is larger than N luma samples (N could be a number equals to power of 2 and larger than 2, i.e 4, 8, 16, 32, 64, 128, . . . );

current block is NOT applied with intra sub-partition (ISP); or

current block is a luma block.

6. The decoder of claim 4 , wherein the programming, when executed by the one or more processors, configures the decoder to perform the method further comprising a position-dependent intra prediction (PDPC) stage, where, during the PDPC stage, reference samples (p[x][y], with x=−1, y=−1 . . . nTbH and x=0 . . . nTbW, y=−1) for the PDPC inputs are not filtered.

7. An encoder comprising:

one or more processors; and

a non-transitory computer-readable storage medium coupled to the processors and storing programming for execution by the one or more processors, wherein the programming, when executed by the one or more processors, configures the encoder to perform a method comprising:

during an intra predicted sample generation stage, obtaining a filtering indication indicating whether a filtered value or an unfiltered value of reference samples of a current coding block is to be used;

in response to determining that a value of the filtering indication is true, generating a prediction sample by using the filtered value of the reference samples based on a planar mode; or

in response to determining that the value of the filtering indication is false, generating a prediction sample by using the unfiltered value of the reference samples based on the planar mode; and

during the intra predicted sample generation stage, obtaining the filtered value of the reference samples by applying a filter to the reference samples, wherein:

when the filter is [1 2 1],

predV[x][y]=((nH−1−y)*((p[x−1][−1]+2*p[x][−1]+p[x+1][−1]+2)>>2)+(y+1)*p[−1][nTbH])<<Log2 (nW),

predH[x][y]=((nW−1−x)*((p[−1][y−1]+2*p[−1][y]+p[−1][y+1]+2)>>2)+(x+1)*p[nTbW][−1])<<Log2 (nH),

wherein predV[x][y] is a vertical component of the prediction sample, predH[x][y] is a horizontal component of the prediction sample, and p[x][y] (x=−1, y=−1 . . . nTbH and x=0 . . . nTbW, y=−1) is the reference sample; and

when the filter is [1 6 1],

predV[x][y]=((nH−1−y)*((p[x−1][−1]+(p[x][−1]*6)+p[x+1][−1]+4)>>3)+(y+1)*p[−1][nTbH])<<Log2 (nW),

predH[x][y]=((nW−1−x)*((p[−1][y−1]+(p[−1][y]*6)+p[−1][y+1]+4)>>3)+(x+1)*p[nTbW][−1])<<Log2 (nH),

wherein predV[x][y] is a vertical component of the prediction sample, predH[x][y] is a horizontal component of the prediction sample, and p[x][y] (x=−1, y=−1 . . . nTbH and x=0 . . . nTbW, y=−1) is the reference sample.

8. The encoder of claim 7 , wherein the value of the filtering indication is true when one or more of the following conditions are true:

current block size is larger than N luma samples (N could be a number equals to power of 2 and larger than 2, i.e 4, 8, 16, 32, 64, 128, . . . );

current block is NOT applied with intra sub-partition (ISP); or

current block is a luma block.

9. The encoder of claim 7 , wherein the programming, when executed by the one or more processors, configures the encoder to perform the method further comprising a position-dependent intra prediction (PDPC) stage, where, during the PDPC stage, reference samples (p[x][y], with x=−1, y=−1 . . . nTbH and x=0 . . . nTbW, y=−1) for the PDPC inputs are not filtered.