Systems and methods for signaling neural network post-filter overlap and sublayer frame rate upsampling information in video coding
A device may be configured to perform filtering based on information included in a neural network post-filter characteristics message. In one example, the neural network post-filter characteristics message includes an instance of syntax element specifying a number of interpolated pictures generated by a post-processing filter for each of a number of sublayers associated with the neural network post-filter characteristics message.
1. A method of performing neural network filtering for video data, the method comprising:
receiving a neural network post-filter characteristics message specifying a neural network that may be used as a post-processing filter;
parsing a nnpfc_constant_patch_size_flag syntax element in the neural network post-filter characteristics message, wherein (i) the nnpfc_constant_patch_size_flag syntax element equal to 1 indicates that the post-processing filter accepts exactly a patch size indicated by a nnpfc_patch_width_minus1 syntax element and a nnpfc_patch_height_minus1 syntax element as input, and (ii) the nnpfc_constant_patch_size_flag syntax element equal to 0 indicates that the post-processing filter accepts as input any patch size with a value of inpPatchWidth+2*overlapSize being a positive integer multiple of an indicated width and a value of inpPatchHeight+2*overlapSize being a positive integer multiple of an indicated height,
wherein,
inpPatch Width is a patch size width,
inpPatchHeight is a patch size height, and
overlapSize is indicated by a nnpfc_overlap syntax element in the neural network post-filter characteristics message, wherein the nnpfc_overlap syntax element indicates overlapping horizontal and vertical sample counts of adjacent input tensors of the post-processing filter.
2. A device comprising one or more processors configured to:
receive a neural network post-filter characteristics message specifying a neural network that may be used as a post-processing filter;
parse a nnpfc_constant_patch_size_flag syntax element in the neural network post-filter characteristics message, wherein (i) the nnpfc_constant_patch_size_flag syntax element equal to 1 indicates that the post-processing filter accepts exactly a patch size indicated by a nnpfc_patch_width_minus1 syntax element and a nnpfc_patch_height_minus1 syntax element as input, and (ii) the nnpfc_constant_patch_size_flag syntax element equal to 0 indicates that the post-processing filter accepts as input any patch size with a value of inpPatchWidth+2*overlapSize being a positive integer multiple of an indicated width and a value of inpPatchHeight+2*overlapSize being a positive integer multiple of an indicated height,
wherein,
inpPatch Width is a patch size width,
inpPatchHeight is a patch size height, and
overlapSize is indicated by a nnpfc_overlap syntax element in the neural network post-filter characteristics message, wherein the nnpfc_overlap syntax element indicates overlapping horizontal and vertical sample counts of adjacent input tensors of the post-processing filter.
3. The device of claim 2 , wherein the device includes a video decoder.
4. A device comprising one or more processors configured to:
signal a neural network post-filter characteristics message specifying a neural network that may be used as a post-processing filter;
signal a nnpfc_constant_patch_size_flag syntax element in the neural network post-filter characteristics message, wherein (i) the nnpfc_constant_patch_size_flag syntax element equal to 1 indicates that the post-processing filter accepts exactly a patch size indicated by a nnpfc_patch_width_minus1 syntax element and a nnpfc_patch_height_minus1 syntax element as input, and (ii) the nnpfc_constant_patch_size_flag syntax element equal to 0 indicates that the post-processing filter accepts as input any patch size with a value of inpPatchWidth+2*overlapSize being a positive integer multiple of an indicated width and a value of inpPatchHeight+2*overlapSize being a positive integer multiple of an indicated height,
wherein,
inpPatch Width is a patch size width,
inpPatchHeight is a patch size height, and
overlapSize is indicated by a nnpfc_overlap syntax element in the neural network post-filter characteristics message, wherein the nnpfc_overlap syntax element indicates overlapping horizontal and vertical sample counts of adjacent input tensors of the post-processing filter.
5. The device of claim 4 , wherein the device includes a video encoder.