Using neural network filtering in video coding

1. A method of processing visual media data, comprising:

determining, for a conversion between visual media data and a bitstream of the visual media data, how to apply one or more convolutional neural network filters to at least some samples of a video unit of the visual media data according to a rule; and

performing the conversion based on the determining,

wherein the rule specifies that a selection of a set of convolutional neural network filters depends on a category of a temporal layer identification of the video unit, wherein a given set of convolutional neural network filters is applied for a corresponding category of the temporal layer identification, and wherein the temporal layer identification is classified to more than one categories based on a GOP (group of pictures) size of the video unit,

wherein the rule further specifies that whether to apply the set of convolutional neural network filters depends on a sample position of a sample within the video unit.

2. The method of claim 1 , wherein the one or more convolutional neural network filter is implemented using a convolutional neural network.

3. The method of claim 1 , wherein the rule specifies that the determining is based on decoded information associated with the video unit, wherein the decoded information includes at least one of prediction modes, transform types, a skip flag, or coded block flag (CBF) values.

4. The method of claim 1 , wherein the rule specifies that information related to the one or more convolutional neural network filters is controlled at a granularity smaller than the video unit.

5. The method of claim 4 , wherein the video unit is a slice or a picture or a tile or a subpicture or a coding tree block or a coding tree unit.

6. The method of claim 4 , wherein the information is controlled at a sample or pixel level.

7. The method of claim 4 , wherein the information is controlled at a row or a column or a line level.

8. The method of claim 4 , wherein the rule specifies that the set of convolutional neural network filters is determined based on a value of a sample within the video unit of the visual media data.

9. The method of claim 1 , wherein the rule specifies that a set of convolutional neural network filters is utilized for video units with different temporal layers.

10. The method of claim 1 , wherein the performing of the conversion comprises generating the bitstream from the visual media data.

11. The method of claim 1 , wherein the performing of the conversion comprises generating the visual media data from the bitstream.

12. An apparatus for processing visual media data comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:

determine, for a conversion between visual media data and a bitstream of the visual media data, how to apply one or more convolutional neural network filters to at least some samples of a video unit of the visual media data according to a rule; and

perform the conversion based on the determining,

wherein the rule specifies that a selection of a set of convolutional neural network filters depends on a category of a temporal layer identification of the video unit, wherein a given set of convolutional neural network filters is applied for a corresponding category of the temporal layer identification, and wherein the temporal layer identification is classified to more than one categories based on a GOP (group of pictures) size of the video unit,

wherein the rule further specifies that whether to apply the set of convolutional neural network filters depends on a sample position of a sample within the video unit.

13. The apparatus of claim 12 , wherein the rule specifies that the determining is based on decoded information associated with the video unit, wherein the decoded information includes at least one of prediction modes, transform types, a skip flag, or coded block flag (CBF) values.

14. The apparatus of claim 12 , wherein the rule specifies that information related to the one or more convolutional neural network filters is controlled at a granularity smaller than the video unit.

15. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

determine, for a conversion between visual media data and a bitstream of the visual media data, how to apply one or more convolutional neural network filters to at least some samples of a video unit of the visual media data according to a rule; and

perform the conversion based on the determining,

wherein the rule specifies that a selection of a set of convolutional neural network filters depends on a category of a temporal layer identification of the video unit, wherein a given set of convolutional neural network filters is applied for a corresponding category of the temporal layer identification, and wherein the temporal layer identification is classified to more than one categories based on a GOP (group of pictures) size of the video unit,

wherein the rule further specifies that whether to apply the set of convolutional neural network filters depends on a sample position of a sample within the video unit.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the rule specifies that the determining is based on decoded information associated with the video unit, wherein the decoded information includes at least one of prediction modes, transform types, a skip flag, or coded block flag (CBF) values.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the rule specifies that information related to the one or more convolutional neural network filters is controlled at a granularity smaller than the video unit.