IMAGE AND VIDEO CODING WIHT ADAPTIVE QUANTIZATION FOR MACHINE-BASED APPLICATIONS
A video coding system for machines employs adaptive quantization based on the frequency response of the machine model receiving the data. The encoder receives the machine model for the machine-based system and generates a frequency importance map from the machine model. An adjustment matrix is then generated based on the frequency importance map and is used to adjust coefficients of the default quantization matrix. The video data is quantized using the adjusted quantization matrix and encoded in a bitstream for transmission to a decoder site. At the decoder site, the decoder can extract parameters of the adjusted quantization matrix from the bitstream or calculate the adjusted quantization matrix using the machine model to inverse quantize the received data for machine consumption.
1 . A video encoder for encoding video data for machines using adaptive quantization, the encoder including a quantization processor having a default quantization matrix and performing the steps comprising:
obtaining a machine model for the machine-based system receiving the video data;
generate a frequency importance map from the machine model;
determine an adjustment matrix based on the frequency importance map;
adjust the default quantization matrix using the adjustment matrix; and
quantize the video data using the adjusted quantization matrix.
2 . The encoder of claim 1 , wherein the frequency importance map is implicitly determined from the machine model by iteratively testing the sensitivity of the model output to the changes in each frequency band of interest.
3 . The encoder of claim 2 , wherein the testing of the frequency sensitivity of the machine model further comprises using a gradient method which is based on the chain rule for differentiation.
4 . The encoder of claim 1 , wherein the frequency importance map is implicitly determined using statistics of the sample dataset on which the machine model is trained.
5 . The encoder of claim 1 , wherein adjusting the default quantization matrix comprises calculating a Hadamard product of the default matrix and the adjustment matrix.
6 . An adaptive quantization module for encoding or decoding video data, the adaptive quantization module having a processor programmed to perform an adaptive quantization method, comprising:
obtaining a machine model for the machine-based system receiving the video data;
generate a frequency importance map from the machine model;
determine an adjustment matrix based on the frequency importance map;
adjust the default quantization matrix using the adjustment matrix; and
quantize the video data using the adjusted quantization matrix.
7 . The adaptive quantization module of claim 6 , wherein the frequency importance map is implicitly determined from the machine model by iteratively testing the sensitivity of the model output to the changes in each frequency band of interest.
8 . The adaptive quantization module of claim 7 , wherein the testing of the frequency sensitivity of the machine model further comprises using a gradient method which is based on the chain rule for differentiation.
9 . The adaptive quantization module of claim 6 , wherein the frequency importance map is implicitly determined using statistics of the sample dataset on which the machine model is trained.
10 . The adaptive quantization module of claim 6 , wherein adjusting the default quantization matrix comprises calculating a Hadamard product of the default matrix and the adjustment matrix.
11 . A decoder for decoding a video bitstream for machine consumption, the decoder having an adaptive quantization module programmed to perform inverse quantization of a bitstream encoded with an adaptive quantization method, the inverse quantization comprising:
obtaining a machine model for the machine-based system receiving the video data;
generate a frequency importance map from the machine model;
determine an adjustment matrix based on the frequency importance map;
adjust the default quantization matrix using the adjustment matrix; and
inverse quantize the video data using the adjusted quantization matrix.
12 . The decoder of claim 11 , wherein the frequency importance map is implicitly determined from the machine model by iteratively testing the sensitivity of the model output to the changes in each frequency band of interest.
13 . The decoder of claim 12 , wherein the testing of the frequency sensitivity of the machine model further comprises using a gradient method which is based on the chain rule for differentiation.
14 . The decoder of claim 11 , wherein the frequency importance map is implicitly determined using statistics of the sample dataset on which the machine model is trained.
15 . The decoder of claim 11 , wherein adjusting the default quantization matrix comprises calculating a Hadamard product of the default matrix and the adjustment matrix.