Encoder, decoder and corresponding methods for adaptive loop filtering

1 . A non-transitory storage medium storing a bitstream and one or more instructions executable by at least one processor to perform operations of encoding or decoding of the bitstream, the operations comprising:

obtaining a reconstructed sample value for a block of a picture;

obtaining filter coefficients for adaptive loop filtering according to the bitstream;

obtaining a sum value according to the filter coefficients, and the reconstructed sample value for the block;

rounding the sum value according to a vertical position of a sample in the block to obtain a rounded sum value, wherein the vertical position of the sample in the block is a luma sample position, and wherein the rounded sum value equals curr+ ((sum+roundY) >>alfShiftY) and satisfies the following table:

Condition

roundY

alfShiftY

y1

y2

y3

(y = = CtbSizeY − 5 | | y = = CtbSizeY − 4) &&

512

10

0

0

0

(applyAlfLineBufBoundary = = 1)

(y = = CtbSizeY − 6 | | y = = CtbSizeY − 3) &&

 64

 7

1

1

1

(applyAlfLineBufBoundary = = 1)

(y = = CtbSizeY − 7 | | y = = CtbSizeY − 2) &&

 64

 7

1

2

2

(applyAlfLineBufBoundary = = 1)

Otherwise

 64

 7

1

2

3

wherein

y represents the vertical position of the sample;

curr represents the reconstructed sample value for the block;

sum represents the sum value;

CtbSizeY represents a size of a coding tree block (CTB) corresponding to the block;

y1, y2 and y3 represent vertical sample position offsets indicating other reconstructed sample values involved in the obtaining the sum value;

roundY and alfShiftY are variables that are determined based on the vertical position of the sample;

apply AlfLineBufBoundary is an index and derived as follows:

if a bottom boundary of the coding tree block is a bottom boundary of the picture and pic_height_in_luma_samples-yCtb <=CtbSizeY- 4 , apply AlfLineBufBoundary is set equal to 0, otherwise, apply AlfLineBufBoundary is set equal to 1,

wherein the pic_height_in_luma_samples represents a height of the picture in luma samples, and (xCtb, yCtb) represents a location of the CTB; and

obtaining a filter reconstructed sample value for the block according to the rounded sum value.

2 . The non-transitory storage medium of claim 1 , wherein roundY is equal to 512 and alfShiftY is equal to 10 when the vertical position of the sample satisfies any one of the below conditions:

y==CtbSizeY−t−1, or

y==CtbSizeY−t, or

y==ctbHeightC−t, or

y==ctbHeightC−t−1,

wherein y is the vertical position of the sample, CtbSizeY is a coding tree block (CTB) size, ctbHeightC is a height of the CTB, and t is an integer value, wherein t is 2, 3, 4, 5 or 6.

3 . The non-transitory storage medium of claim 1 , wherein roundY is equal to 64 and alfShiftY is equal to 7 when the vertical position of the sample does not satisfy any one of the below conditions:

y==CtbSizeY−t−1, or

y==CtbSizeY−t, or

y==ctbHeightC−t, or

y==ctbHeightC−t−1,

wherein y is the vertical position of the sample, CtbSizeY is a coding tree block (CTB) size, ctbHeightC is a height of the CTB, and t is an integer value, wherein t is 2, 3, 4, 5 or 6.

4 . The non-transitory storage medium of claim 1 , wherein when the vertical position of the sample in the block is equal to 0, alfShiftY is equal to 10.

5 . The non-transitory storage medium of claim 1 , wherein when the vertical position of the sample in the block is not equal to 0, alfShiftY is equal to 7.

6 . The non-transitory storage medium of claim 1 , wherein the rounding the sum value comprises:

adding the value roundY which is determined based on the vertical position of the sample to the sum value in order to obtain an added sum value, and shifting the added sum value based on the vertical position of the sample.

7 . A non-transitory storage medium storing instructions executable by a decoder to perform operations comprising:

obtaining a reconstructed sample value for a block of a picture;

obtaining filter coefficients for adaptive loop filtering according to a bitstream;

obtaining a sum value according to the filter coefficients and the reconstructed sample value for the block;

rounding the sum value according to a vertical position of a sample in the block, to obtain a rounded sum value, wherein the vertical position of the sample in the block is a luma sample position, and wherein the rounded sum value equals curr+((sum+roundY) >alfShiftY) and satisfies the following table:

Condition

roundY

alfShiftY

y1

y2

y3

(y = = CtbSizeY − 5 | | y = = CtbSizeY − 4) &&

512

10

0

0

0

(applyAlfLineBufBoundary = = 1)

(y = = CtbSizeY − 6 | | y = = CtbSizeY − 3) &&

 64

 7

1

1

1

(applyAlfLineBufBoundary = = 1)

(y = = CtbSizeY − 7 y = = CtbSizeY − 2) &&

 64

 7

1

2

2

(applyAlfLineBufBoundary = = 1)

Otherwise

 64

 7

1

2

3

wherein

y represents the vertical position of the sample;

curr represents the reconstructed sample value for the block;

sum represents the sum value;

CtbSizeY represents a size of a coding tree block (CTB) corresponding to the block;

y1, y2 and y3 represent vertical sample position offsets indicating other reconstructed sample values involved in the obtaining the sum value;

roundY and alfShiftY are variables that are determined based on the vertical position of the sample;

applyAlfLineBufBoundary an index and is derived as follows:

if a bottom boundary of the coding tree block is a bottom boundary of the picture and pic_height_in_luma_samples-yCtb <=CtbSizeY 4, applyAlfLineBufBoundary is set equal to 0, otherwise, applyAlfLineBufBoundary is set equal to 1,

wherein the pic_height_in_luma_samples represents a height of the picture in luma samples, and (xCtb, yCtb) represents a location of the CTB; and

obtaining a filter reconstructed sample value for the block according to the rounded sum value.

8 . The non-transitory storage medium of claim 7 , wherein roundY is equal to 512 and alfShiftY is equal to 10 when the vertical position of the sample satisfies any one of the below conditions:

y==CtbSizeY−t−1, or

y==CtbSizeY−t, or

y==ctbHeightC−t, or

y==ctbHeightC−t−1,

wherein y is the vertical position of the sample, CtbSizeY is a coding tree block (CTB) size, ctbHeightC is a height of the CTB, and t is an integer value, wherein t is 2, 3, 4, 5 or 6.

9 . The non-transitory storage medium of claim 7 , wherein roundY is equal to 64 and alfShiftY is equal to 7 when the vertical position of the sample does not satisfy any one of the below conditions:

y==CtbSizeY−t−1, or

y==CtbSizeY−t, or

y==ctbHeightC−t, or

y==ctbHeightC-t−1,

wherein y is the vertical position of the sample, CtbSizeY is a coding tree block (CTB) size, ctbHeightC is a height of the CTB, and t is an integer value, wherein t is 2, 3, 4, 5 or 6.

10 . The non-transitory storage medium of claim 7 , wherein when the vertical position of the sample in the block is equal to 0, alfShiftY is equal to 10.

11 . The non-transitory storage medium of claim 7 , wherein when the vertical position of the sample in the block is not equal to 0, alfShiftY is equal to 7.

12 . The non-transitory storage medium of claim 7 , wherein the rounding the sum value comprises:

adding the value roundY which is determined based on the vertical position of the sample to the sum value in order to obtain an added sum value, and

shifting the added sum value based on the vertical position of the sample.