Sample padding for cross-component adaptive loop filtering

1. A method of processing video data, comprising:

determining, for a conversion between a video unit of a video and a bitstream of the video, that a mirrored padding process for padding an unavailable luma sample, which is unavailable due to a virtual boundary, is used during an application of a cross-component adaptive loop filtering (CC-ALF) tool to the video unit when the virtual boundary is applied to the video unit; and

performing the conversion based on the determining,

wherein the mirrored padding process includes padding a corresponding sample of the unavailable luma sample and considering the corresponding sample as unavailable even when the corresponding sample is available for the virtual boundary, whereby the corresponding sample is located in a filter support region in the CC-ALF tool,

wherein a row position of the corresponding sample to be padded meets a distance condition, wherein the distance condition is related to a first distance and a second distance, wherein the first distance is a distance between a row where a representative luma sample is located in the video unit and a row where the unavailable luma sample is located, wherein the second distance is a distance between a row where the representative luma sample is located and a row where a corresponding luma sample is located, and wherein the representative luma sample is a collocated luma sample of a chroma sample to be filtered in the video unit.

2. The method of claim 1 , wherein the unavailable luma sample is denoted as located at row M in the video unit and the corresponding sample to be padded is denoted as located at row N in the video unit, and wherein distance condition is d (C, M)=d (N, C), whereby d (x, y) denotes a distance between row x and row y, and wherein C represents a row in which the representative luma sample is located in the video unit, where C, M, and N are integers and M is not equal to N.

3. The method of claim 1 , wherein a position of the collocated luma sample of the chroma sample depends on a color format of the video.

4. The method of claim 3 , wherein the collocated luma sample of the chroma sample located at (x, y) is defined as a sample located at (2x, 2y) when the color format is 4:2:0, the sample located at (2x, y) when the color format is 4:2:2, or the sample located at (x, y) when the color format is 4:4:4.

5. The method of claim 1 , wherein the CC-ALF tool has a K×L filter shape, whereby K=4 and L=3; yM1, yL, yP1, yP2 are y-coordinates of four sample rows in the K×L filter shape, whereby yM1=−1, yL=0, yP1=1, yP2=2, the representative luma sample is in a row yL=0; and the virtual boundary is located at a row CtbSizeY−4 in the video unit, whereby CtbSizeY represents a size of the video unit which is a coding tree unit (CTU).

6. The method of claim 5 , wherein the unavailable luma sample is padded using a nearest available row below the virtual boundary when the virtual boundary is above the representative luma sample, and wherein the corresponding sample is padded using a nearest available row above a row containing the corresponding sample.

7. The method of claim 6 , wherein, when yL is equal to CtbSizeY−3 and a row above yM1 is unavailable, a corresponding sample (x, yP2) in a row at yP2 is padded using a sample (x, yP1) in a row at yP1.

8. The method of claim 6 , wherein, when yL is equal to CtbSizeY−4 and a row above yM1 and a row at yM1 are unavailable, an unavailable luma sample (x, yM1) in a row at yM1 is padded using a sample (x, yL) in a row at yL, a corresponding sample (x, yP1) in a row at yP1 is padded using a sample (x, yL) in a row at yL, and a corresponding sample (x, yP2) in a row at yP2 is padded using a sample (x, yL) in a row at yL.

9. The method of claim 5 , wherein the unavailable luma sample is padded using a nearest available row above the virtual boundary when the virtual boundary is below the representative luma sample, and wherein the corresponding sample is padded using a nearest available row below a row containing the corresponding sample.

10. The method of claim 9 , wherein when yL is equal to CtbSizeY−6 and a row at yP2 is unavailable, an unavailable luma sample (x, yP2) in the row at yP2 is padded using a sample (x, yP1) in a row at yP1.

11. The method of claim 9 , wherein, when yL is equal to CtbSizeY−5 and rows at yP2 and yP1 are unavailable, an unavailable luma sample (x, yP2) at a row yP2 is padded using a sample (x, yL) in a row at yL, an unavailable luma sample (x, yP1) at a row yP1 is padded using the sample (x, yL) in the row at yL, and a corresponding sample (x, yM1) in a row at yM1 is padded using the sample (x, yL) in the row at yL.

12. The method of claim 1 , wherein the conversion includes encoding the video into the bitstream.

13. The method of claim 1 , wherein the conversion includes decoding the video from the bitstream.

14. An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, for a conversion between a video unit of a video and a bitstream of the video, that a mirrored padding process for padding an unavailable luma sample, which is unavailable due to a virtual boundary, is used during an application of a cross-component adaptive loop filtering (CC-ALF) tool to the video unit when the virtual boundary is applied to the video unit; and

perform the conversion based on the determination,

wherein the mirrored padding process includes padding a corresponding sample of the unavailable luma sample and considering the corresponding sample as unavailable even when the corresponding sample is available for the virtual boundary, whereby the corresponding sample is located in a filter support region in the CC-ALF tool,

wherein a row position of the corresponding sample to be padded meets a distance condition, wherein the distance condition is related to a first distance and a second distance, wherein the first distance is a distance between a row where a representative luma sample is located in the video unit and a row where the unavailable luma sample is located, wherein the second distance is a distance between a row where the representative luma sample is located and a row where a corresponding luma sample is located, and wherein the representative luma sample is a collocated luma sample of a chroma sample to be filtered in the video unit.

15. The apparatus of claim 14 , wherein the unavailable luma sample is denoted as located at row M in the video unit and the corresponding sample to be padded is denoted as located at row N in the video unit, and wherein distance condition is d (C, M)=d (N, C), whereby d (x, y) denotes a distance between row x and row y, wherein C represents a row where the representative luma sample is located in the video unit, where C, M, and N are integers and M is not equal to N;

wherein a position of the collocated luma sample of the chroma sample depends on a color format of the video; and

wherein the collocated luma sample of the chroma sample located at (x, y) is defined as a sample located at (2x, 2y) when the color format is 4:2:0, the sample located at (2x, y) when the color format is 4:2:2, or the sample located at (x, y) when the color format is 4:4:4.

16. The apparatus of claim 14 , wherein the CC-ALF tool has a K×L filter shape, whereby K=4 and L=3; yM1, yL, yP1, yP2 are y-coordinates of four sample rows in the K×L filter shape, whereby yM1=−1, yL=0, yP1=1, yP2=2, the representative luma sample is in a row yL=0; and the virtual boundary is located at a row CtbSizeY−4 in the video unit, whereby CtbSizeY represents a size of the video unit which is a coding tree unit (CTU);

wherein the unavailable luma sample is padded using a nearest available row below the virtual boundary when the virtual boundary is above the representative luma sample wherein the corresponding sample is padded using a nearest available row above a row containing the corresponding sample, wherein, when yL is equal to CtbSizeY−3 and a row above yM1 is unavailable, a corresponding sample (x, yP2) in a row at yP2 is padded using a sample (x, yP1) in a row at yP1; or wherein, when yL is equal to CtbSizeY−4 and a row above yM1 and a row at yM1 are unavailable, an unavailable luma sample (x, yM1) in a row at yM1 is padded using a sample (x, yL) in a row at yL, a corresponding sample (x, yP1) in a row at yP1 is padded using a sample (x, yL) in a row at yL, and a corresponding sample (x, yP2) in a row at yP2 is padded using a sample (x, yL) in a row at yL;

wherein the unavailable luma sample is padded using a nearest available row above the virtual boundary when the virtual boundary is below the representative luma sample and wherein the corresponding sample is padded using a nearest available row below a row containing the corresponding sample, wherein when yL is equal to CtbSizeY−6 and a row at yP2 is unavailable, the unavailable luma sample (x, yP2) in the row at yP2 is padded using a sample (x, yP1) in a row at yP1; and

wherein, when yL is equal to CtbSizeY−5 and rows at yP2 and yP1 are unavailable, an unavailable luma sample (x, yP2) at a row yP2 is padded using a sample (x, yL) in a row at yL, an unavailable luma sample (x, yP1) at a row yP1 is padded using the sample (x, yL) in the row at yL, and a corresponding sample (x, yM1) in a row at yM1 is padded using the sample (x, yL) in the row at yL.

17. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, for a conversion between a video unit of a video and a bitstream of the video, that a mirrored padding process for padding an unavailable luma sample which is unavailable due to a virtual boundary is used during an application of a cross-component adaptive loop filtering (CC-ALF) tool to the video unit when the virtual boundary is applied to the video unit; and

perform the conversion based on the determination,

wherein the mirrored padding process includes padding a corresponding sample of the unavailable luma sample and considering the corresponding sample as unavailable even when the corresponding sample is available for the virtual boundary, whereby the corresponding sample is located in a filter support region in the CC-ALF tool,

wherein a row position of the corresponding sample to be padded meets a distance condition, wherein the distance condition is related to a first distance and a second distance, wherein the first distance is a distance between a row where a representative luma sample is located in the video unit and a row where the unavailable luma sample is located, wherein the second distance is a distance between a row where the representative luma sample is located and a row where a corresponding luma sample is located, and wherein the representative luma sample is a collocated luma sample of a chroma sample to be filtered in the video unit.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the unavailable luma sample is denoted as located at row M in the video unit and the corresponding sample to be padded is denoted as located at row N in the video unit, and wherein distance condition is d (C, M)=d (N, C), whereby d (x, y) denotes a distance between row x and row y, wherein C represents a row where the representative luma sample is located in the video unit, where C, M and N are integers and M is not equal to N;

wherein a position of the collocated luma sample of the chroma sample depends on a color format of the video;

wherein the collocated luma sample of the chroma sample located at (x, y) is defined as a sample located at (2x, 2y) when the color format is 4:2:0, or the sample located at (2x, y) when the color format is 4:2:2, or the sample located at (x, y) when the color format is 4:4:4; wherein the CC-ALF tool has a K×L filter shape, whereby K=4 and L=3; yM1, yL, yP1, yP2 are y-coordinates of four sample rows in the K×L filter shape, whereby yM1=−1, yL=0, yP1=1, yP2=2, the representative luma sample is in a row yL=0; and the virtual boundary is located at a row CtbSizeY−4 in the video unit, whereby CtbSizeY represents a size of the video unit which is a coding tree unit (CTU);

wherein the unavailable luma sample is padded using a nearest available row below the virtual boundary when the virtual boundary is above the representative luma sample and wherein the corresponding sample is padded using a nearest available row above a row containing the corresponding sample, wherein, when yL is equal to CtbSizeY−3 and a row above yM1 is unavailable, a corresponding sample (x, yP2) in a row at yP2 is padded using a sample (x, yP1) in a row at yP1; or wherein, when yL is equal to CtbSizeY−4 and a row above yM1 and a row at yM1 are unavailable, an unavailable luma sample (x, yM1) in a row at yM1 is padded using a sample (x, yL) in a row at yL, a corresponding sample (x, yP1) in a row at yP1 is padded using a sample (x, yL) in a row at yL, and a corresponding sample (x, yP2) in a row at yP2 is padded using a sample (x, yL) in a row at yL;

wherein the unavailable luma sample is padded using a nearest available row above the virtual boundary when the virtual boundary is below the representative luma sample and wherein the corresponding sample is padded using a nearest available row below a row containing the corresponding sample, wherein when yL is equal to CtbSizeY−6 and a row at yP2 is unavailable, the unavailable luma sample (x, yP2) in the row at yP2 is padded using a sample (x, yP1) in a row at yP1; and

wherein, when yL is equal to CtbSizeY−5 and rows at yP2 and yP1 are unavailable, an unavailable luma sample (x, yP2) at a row yP2 is padded using a sample (x, yL) in a row at yL, an unavailable luma sample (x, yP1) at a row yP1 is padded using the sample (x, yL) in the row at yL, and a corresponding sample (x, yM1) in a row at yM1 is padded using the sample (x, yL) in the row at yL.

19. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:

determining that a mirrored padding process for padding an unavailable luma sample which is unavailable due to a virtual boundary is used during an application of a cross-component adaptive loop filtering (CC-ALF) tool to a video unit of a video when the virtual boundary is applied to the video unit; and

generating the bitstream based on the determining,

wherein the mirrored padding process includes padding a corresponding sample of the unavailable luma sample and considering the corresponding sample as unavailable even when the corresponding sample is available for the virtual boundary, whereby the corresponding sample is located in a filter support region in the CC-ALF tool,

wherein a row position of the corresponding sample to be padded meets a distance condition, the distance condition is related to a first distance and a second distance, wherein the first distance is a distance between a row where a representative luma sample is located in the video unit and a row where the unavailable luma sample is located, wherein the second distance is a distance between a row where the representative luma sample is located and a row where a corresponding luma sample is located, and wherein the representative luma sample is a collocated luma sample of a chroma sample to be filtered in the video unit.

20. The non-transitory computer-readable recording medium of claim 19 ,

wherein the unavailable luma sample is denoted as located at row M in the video unit and the corresponding sample to be padded is denoted as located at row N in the video unit, and wherein distance condition is d (C, M)=d (N, C), whereby d (x, y) denotes a distance between row x and row y, wherein C represents a row where the representative luma sample is located in the video unit, where C, M and N are integers and M is not equal to N;

wherein a position of the collocated luma sample of the chroma sample depends on a color format of the video;

wherein the collocated luma sample of the chroma sample located at (x, y) is defined as a sample located at (2x, 2y) when the color format is 4:2:0, the sample located at (2x, y) when the color format is 4:2:2, or the sample located at (x, y) when the color format is 4:4:4;

wherein the CC-ALF tool has a K×L filter shape, whereby K=4 and L=3; yM1, yL, yP1, yP2 are y-coordinates of four sample rows in the K×L filter shape, whereby yM1=−1, yL=0, yP1=1, yP2=2, the representative luma sample is in a row yL=0; and the virtual boundary is located at a row CtbSizeY−4 in the video unit, whereby CtbSizeY represents a size of the video unit which is a coding tree unit (CTU);

wherein the unavailable luma sample is padded using a nearest available row below the virtual boundary when the virtual boundary is above the representative luma sample and wherein the corresponding sample is padded using a nearest available row above a row containing the corresponding sample, wherein, when yL is equal to CtbSizeY−3 and a row above yM1 is unavailable, a corresponding sample (x, yP2) in a row at yP2 is padded using a sample (x, yP1) in a row at yP1; or wherein, when yL is equal to CtbSizeY−4 and a row above yM1 and a row at yM1 are unavailable, an unavailable luma sample (x, yM1) in a row at yM1 is padded using a sample (x, yL) in a row at yL, a corresponding sample (x, yP1) in a row at yP1 is padded using a sample (x, yL) in a row at yL, and a corresponding sample (x, yP2) in a row at yP2 is padded using a sample (x, yL) in a row at yL;

wherein the unavailable luma sample is padded using a nearest available row above the virtual boundary when the virtual boundary is below the representative luma sample and wherein the corresponding sample is padded using a nearest available row below a row containing the corresponding sample, wherein when yL is equal to CtbSizeY−6 and a row at yP2 is unavailable, the unavailable luma sample (x, yP2) in the row at yP2 is padded using a sample (x, yP1) in a row at yP1; and

wherein, when yL is equal to CtbSizeY−5 and rows at yP2 and yP1 are unavailable, an unavailable luma sample (x, yP2) at a row yP2 is padded using a sample (x, yL) in a row at yL, an unavailable luma sample (x, yP1) at a row yP1 is padded using the sample (x, yL) in the row at yL, and a corresponding sample (x, yM1) in a row at yM1 is padded using the sample (x, yL) in the row at yL.