Adaptive quantization for neural network weights for convolution neural network filters in video coding

1 . A method of decoding video data, the method comprising:

determining first neural network (NN) weights for a layer of a convolutional neural network (CNN) filter;

deriving a quantization value for the layer of the CNN filter based on values of the first NN weights, wherein deriving the quantization value for the layer of the CNN filter based on the values of the first NN weights comprises testing a candidate quantization value to determine an amount of clipping produced by the candidate quantization value;

converting the first NN weights to second NN weights based on the quantization value;

filtering a block of the video data using the second NN weights to produce a filtered block of video data; and

outputting a picture of decoded video data that includes the filtered block.

2 . The method of claim 1 , wherein deriving the quantization value for the layer of the CNN filter based on the values of the first NN weights comprises testing a candidate quantization value to determine a maximum difference between an original parameter value and a quantized-then-dequantized version of the original parameter value produced by the candidate quantization value.

3 . The method of claim 1 , wherein deriving the quantization value for the layer of the CNN filter based on the values of the first NN weights comprises testing a candidate quantization value to determine a number of values that are clipped due to overflow and a magnitude of clipping errors produced by the candidate quantization value.

4 . The method of claim 1 , wherein deriving the quantization value for the layer of the CNN filter based on the values of the first NN weights comprises testing a candidate quantization value to determine a percentage of parameters that overflow produced by the candidate quantization value.

5 . The method of claim 1 , wherein deriving the quantization value for the layer of the CNN filter based on the values of the first NN weights comprises testing a candidate quantization value to determine a percentage of parameters that overflow as a result of the candidate quantization value.

6 . The method of claim 1 , wherein

deriving the quantization value for the layer of the CNN filter based on the values of the first NN weights comprises testing a candidate quantization value to determine a quantization error produced by the candidate quantization value and a clipping error produced by the candidate quantization value,

the quantization error corresponds to a maximum difference between an original parameter value and a quantized-then-dequantized version of the original parameter value produced by the candidate quantization value,

the clipping error corresponds to one or more of an amount of clipping produced by the candidate quantization value or a percentage of parameters that overflow as a result of the candidate quantization value.

7 . The method of claim 1 , wherein deriving the quantization value for the layer of the CNN filter based on the values of the first NN weights comprises testing a plurality of candidate quantization values to determine the quantization value.

8 . The method of claim 1 , wherein the first NN weights comprise floating point precision values and the second NN weights comprise integer precision values.

9 . The method of claim 1 , wherein the method of decoding is performed as part of a video encoding process.

10 . A device for decoding video data, the device comprising:

a memory configured to store video data;

one or more processors implemented in circuitry and configured to:

determine first neural network (NN) weights for a layer of a convolutional neural network (CNN) filter;

derive a quantization value for the layer of the CNN filter based on values of the first NN weights, wherein to derive the quantization value for the layer of the CNN filter based on the values of the first NN weights, the one or more processors are further configured to test a candidate quantization value to determine an amount of clipping produced by the candidate quantization value;

convert the first NN weights to second NN weights based on the quantization value;

filter a block of the video data using the second NN weights to produce a filtered block of video data; and

output a picture of decoded video data that includes the filtered block.

11 . The device of claim 10 , wherein to derive the quantization value for the layer of the CNN filter based on the values of the first NN weights, the one or more processors are further configured to test a candidate quantization value to determine a maximum difference between an original parameter value and a quantized-then-dequantized version of the original parameter value produced by the candidate quantization value.

12 . The device of claim 10 , wherein to derive the quantization value for the layer of the CNN filter based on the values of the first NN weights, the one or more processors are configured to test a candidate quantization value to determine a number of values that are clipped due to overflow and a magnitude of clipping errors produced by the candidate quantization value.

13 . The device of claim 10 , wherein to derive the quantization value for the layer of the CNN filter based on the values of the first NN weights, the one or more processors are further configured to test a candidate quantization value to determine a percentage of parameters that overflow produced by the candidate quantization value.

14 . The device of claim 10 , wherein to derive the quantization value for the layer of the CNN filter based on the values of the first NN weights, the one or more processors are further configured to test a candidate quantization value to determine a percentage of parameters that overflow as a result of the candidate quantization value.

15 . The device of claim 10 , wherein

to derive the quantization value for the layer of the CNN filter based on the values of the first NN weights, the one or more processors are further configured to test a candidate quantization value to determine a quantization error produced by the candidate quantization value and a clipping error produced by the candidate quantization value,

the quantization error corresponds to a maximum difference between an original parameter value and a quantized-then-dequantized version of the original parameter value produced by the candidate quantization value, and

the clipping error corresponds to one or more of an amount of clipping produced by the candidate quantization value or a percentage of parameters that overflow as a result of the candidate quantization value.

16 . The device of claim 10 , wherein to derive the quantization value for the layer of the CNN filter based on the values of the first NN weights, the one or more processors are further configured to test a plurality of candidate quantization values to determine the quantization value.

17 . The device of claim 10 , wherein the first NN weights comprise floating point precision values and the second NN weights comprise integer precision values.

18 . A method of encoding video data, the method comprising:

determining first neural network (NN) weights for a layer of a convolutional neural network (CNN) filter;

deriving a quantization value for the layer of the CNN filter based on values of the first NN weights, wherein deriving the quantization value for the layer of the CNN filter based on the values of the first NN weights comprises testing a candidate quantization value to determine an amount of clipping produced by the candidate quantization value;

converting the first NN weights to second NN weights based on the quantization value;

filtering a first block of the video data using the second NN weights to produce a filtered block of the video data;

storing a picture of decoded video data that includes the filtered block; and

predicting a second block of the video data based on the stored picture.