Network based image filtering for video coding
A method and an apparatus for image filtering in video coding using a neural network are provided. The method includes: loading, a plurality of quantization parameter (QP) map (QpMap) values at a plurality of QpMap channels into the neural network; obtaining a QP scaling factor by adjusting a plurality of input QP values related to an input frame; and adjusting, according to a QP scaling factor, the plurality of QpMap values for the neural network to learn and filter the input frame to the neural network.
1 . A method for image filtering in video coding, comprising:
loading a plurality of quantization parameter (QP) map (QpMap) values at one or more QpMap channels into a neural network;
obtaining a QP scaling factor by adjusting a plurality of input QP values related to an input frame; and
adjusting, according to the QP scaling factor, the plurality of QpMap values for the neural network to learn and filter the input frame to the neural network.
2 . The method of claim 1 , wherein adjusting the plurality of input QP values related to the input frame comprises:
obtaining a QP offset based on a QP offset step size, a lower bound, and an offset index, wherein the QP offset step size is a step size for adjusting each of the plurality of input QP values, the lower bound is an integer value determining a maximum QP value reduction, and the offset index is a signaled index value; and
subtracting the QP offset from a QP input value.
3 . The method of claim 2 , further comprising:
signaling, by an encoder, the QP offset step size, the low bound, and the offset index, wherein the offset index is an integer between 0 and 3.
4 . The method of claim 2 , further comprising:
predefining, by an encoder, the QP offset step size and the low bound; and
signaling, by the encoder, the offset index, wherein the offset index is an integer between 0 and 3.
5 . The method of claim 1 , wherein obtaining the QP scaling factor comprises obtaining a QP offset; and
adjusting the plurality of QpMap values comprises subtracting the QP offset from a QP input value.
6 . The method of claim 1 , wherein obtaining the QP scaling factor comprises:
obtaining a QP offset; and
obtaining the QP scaling factor using a following operation:
α CH =2 (-QP offset )/6
wherein α CH indicates the QP scaling factor, and QP offset indicates the QP offset.
7 . The method of claim 6 , further comprising:
determining the QP offset per input video block.
8 . The method of claim 6 , further comprising:
signaling, by an encoder, the QP offset at a frame level.
9 . The method of claim 5 , further comprising:
predefining, by an encoder, a look up table (LUT), wherein the LUT comprises a plurality of code words and a plurality of QP offsets corresponding to the plurality of code words; and
signaling, by the encoder, a code word so that a decoder retrieves the QP offset corresponding to the code word based on the LUT.
10 . The method of claim 9 , further comprising:
predefining, by an encoder, a plurality of look up tables (LUTs) at different granularities; and
signaling, by the encoder, the code word so that the decoder retrieves the QP offset corresponding to the code word based on the plurality of LUTs at different granularities.
11 . The method of claim 9 , further comprising:
predefining, by an encoder, a plurality of look up tables (LUTs) at different granularities; and
signaling, by the encoder, the code word and a LUT step size difference so that the decoder retrieves the QP offset corresponding to the code word based on the code word and the LUT step size difference, wherein the LUT step size difference is a difference between a first step size and a second step size, the first step size is a difference between two adjacent QP offsets in a first LUT, and the second step size is a difference between two adjacent QP offsets in a second LUT.
12 . An apparatus for image filtering in video coding using a neural network, comprising:
one or more processors; and
a memory coupled to the one or more processors and configured to store instructions executable by the one or more processors and a bitstream,
wherein the one or more processors, upon execution of the instructions, are configured to perform operations to generate the bitstream, the operations comprising:
loading a plurality of quantization parameter (QP) map (QpMap) values at one or more QpMap channels into a neural network;
obtaining a QP scaling factor by adjusting a plurality of input QP values related to an input frame; and
adjusting, according to the QP scaling factor, the plurality of QpMap values for the neural network to learn and filter the input frame to the neural network.
13 . The apparatus of claim 12 , wherein adjusting the plurality of input QP values related to the input frame comprises:
obtaining a QP offset based on a QP offset step size, a lower bound, and an offset index, wherein the QP offset step size is a step size for adjusting each of the plurality of input QP values, the lower bound is an integer value determining a maximum QP value reduction, and the offset index is a signaled index value; and
subtracting the QP offset from a QP input value.
14 . The apparatus of claim 13 , wherein the operations further comprise:
signaling, by an encoder, the QP offset step size, the low bound, and the offset index, wherein the offset index is an integer between 0 and 3.
15 . The apparatus of claim 13 , wherein the operations further comprise:
predefining, by an encoder, the QP offset step size and the low bound; and
signaling, by the encoder, the offset index, wherein the offset index is an integer between 0 and 3.
16 . The apparatus of claim 12 , wherein obtaining the QP scaling factor comprises obtaining a QP offset; and
adjusting the plurality of QpMap values comprises subtracting the QP offset from a QP input value.
17 . The apparatus of claim 12 , wherein obtaining the QP scaling factor comprises:
obtaining a QP offset; and
obtaining the QP scaling factor using a following operation:
α CH =2 (-QP offset )/6
wherein α CH indicates the QP scaling factor, and QP offset indicates the QP offset.
18 . The apparatus of claim 17 , wherein the operations further comprise:
determining the QP offset per input video block.
19 . The apparatus of claim 17 , further comprising:
signaling, by an encoder, the QP offset at a frame level.
20 . A non-transitory computer-readable storage medium storing computer-executable instructions and a bitstream, wherein the computer-executable instructions, when executed by one or more computer processors, cause the one or more computer processors to store the bitstream and to perform operations to generate the bitstream, the operations comprising:
loading a plurality of quantization parameter (QP) map (QpMap) values at one or more QpMap channels into a neural network;
obtaining a QP scaling factor by adjusting a plurality of input QP values related to an input frame; and
adjusting, according to the QP scaling factor, the plurality of QpMap values for the neural network to learn and filter the input frame to the neural network.