Restrictions on in-loop filtering
Devices, systems and methods for digital video coding. A method for video processing includes determining whether to apply and how to apply a filtering process for samples of a current video unit of a video based on whether the samples are located at predefined transform boundaries; and performing a conversion between the video and a bitstream representation of the video based on the determining, where the predefined transform boundaries are determined based on a first transform mode in which a transform region is split into multiple transform sub-regions and only one transform sub-region has a non-zero residual sample.
1 . A method of video processing, comprising:
determining whether to apply and how to apply a deblocking filtering process for samples of a current video unit of a video based on whether the samples are located at predefined transform boundaries, wherein at least a boundary of the predefined transform boundaries is filtered in the deblocking filtering process based on the samples being located at the predefined transform boundaries; and
performing a conversion between the video and a bitstream of the video based on the determining,
wherein the predefined transform boundaries are determined based on a sub-block transform mode, wherein, in the sub-block transform mode, a transform region is split into multiple transform sub-regions and only one transform sub-region has one or more transform coefficients,
wherein a size of the only one transform sub-region having the one or more transform coefficients is smaller than or equal to a size of another transform sub-region among the multiple transform sub-regions, and
wherein how to apply a clipping operation on filtered samples is determined based on the predefined transform boundaries.
2 . The method of claim 1 , wherein the deblocking filtering process is applied based on the samples being located at the predefined transform boundaries.
3 . The method of claim 2 , wherein at least one of the samples is non-zero.
4 . The method of claim 1 , wherein a boundary strength value is determined based on the predefined transform boundaries.
5 . The method of claim 1 , wherein the conversion includes encoding the video into the bitstream.
6 . The method of claim 1 , wherein the conversion includes decoding the video from the bitstream.
7 . An apparatus for processing video data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
determine whether to apply and how to apply a deblocking filtering process for samples of a current video unit of a video based on whether the samples are located at predefined transform boundaries, wherein at least a boundary of the predefined transform boundaries is filtered in the deblocking filtering process based on the samples being located at the predefined transform boundaries; and
perform a conversion between the video and a bitstream of the video based on the determining,
wherein the predefined transform boundaries are determined based on a sub-block transform mode, wherein, in the sub-block transform mode, a transform region is split into multiple transform sub-regions and only one transform sub-region has one or more transform coefficients,
wherein a size of the only one transform sub-region having the one or more transform coefficients is smaller than or equal to a size of another transform sub-region among the multiple transform sub-regions, and
wherein how to apply a clipping operation on filtered samples is determined based on the predefined transform boundaries.
8 . The apparatus of claim 7 , wherein the deblocking filtering process is applied based on the samples being located at the predefined transform boundaries.
9 . The apparatus of claim 8 , wherein at least one of the samples is non-zero.
10 . The apparatus of claim 7 , wherein a boundary strength value is determined based on the predefined transform boundaries.
11 . The apparatus of claim 7 , wherein the conversion includes encoding the video into the bitstream.
12 . The apparatus of claim 7 , wherein the conversion includes decoding the video from the bitstream.
13 . A non-transitory computer-readable storage medium storing instructions that cause a processor to:
determine whether to apply and how to apply a deblocking filtering process for samples of a current video unit of a video based on whether the samples are located at predefined transform boundaries, wherein at least a boundary of the predefined transform boundaries is filtered in the deblocking filtering process based on the samples being located at the predefined transform boundaries; and
perform a conversion between the video and a bitstream of the video based on the determining,
wherein the predefined transform boundaries are determined based on a sub-block transform mode, wherein, in the sub-block transform mode, a transform region is split into multiple transform sub-regions and only one transform sub-region has one or more transform coefficients,
wherein a size of the only one transform sub-region having the one or more transform coefficients is smaller than or equal to a size of another transform sub-region among the multiple transform sub-regions, and
wherein how to apply a clipping operation on filtered samples is determined based on the predefined transform boundaries.
14 . The non-transitory computer-readable storage medium of claim 13 , wherein the deblocking filtering process is applied based on the samples being located at the predefined transform boundaries, wherein at least one of the samples is non-zero.
15 . A method for storing a bitstream of a video, comprising:
determining whether to apply and how to apply a deblocking filtering process for samples of a current video unit of the video based on whether the samples are located at predefined transform boundaries, wherein at least a boundary of the predefined transform boundaries is filtered in the deblocking filtering process based on the samples being located at the predefined transform boundaries;
generating the bitstream of the video based on the determining; and
storing the bitstream in a non-transitory computer-readable recording medium,
wherein the predefined transform boundaries are determined based on a sub-block transform mode, wherein, in the sub-block transform mode, a transform region is split into multiple transform sub-regions and only one transform sub-region has one or more transform coefficients,
wherein a size of the only one transform sub-region having the one or more transform coefficients is smaller than or equal to a size of another transform sub-region among the multiple transform sub-regions, and
wherein how to apply a clipping operation on filtered samples is determined based on the predefined transform boundaries.
16 . The method of claim 15 , wherein the deblocking filtering process is applied based on the samples being located at the predefined transform boundaries, wherein at least one of the samples is non-zero.
17 . The non-transitory computer-readable storage medium of claim 13 , wherein a boundary strength value is determined based on the predefined transform boundaries.
18 . The non-transitory computer-readable storage medium of claim 13 , wherein the conversion includes encoding the video into the bitstream.
19 . The non-transitory computer-readable storage medium of claim 13 , wherein the conversion includes decoding the video from the bitstream.
20 . The method of claim 15 , wherein a boundary strength value is determined based on the predefined transform boundaries.