Method of bit allocation for image and video compression using perceptual guidance
View Patent ↗A video processing system is provided to create quantization data parameters based on human eye attraction to provide to an encoder to enable the encoder to compress data taking into account the human perceptual guidance. The system includes a perceptual video processor (PVP) to generate a perceptual significance pixel map for data to be input to the encoder. Companding is provided to reduce the pixel values to values ranging from zero to one, and decimation is performed to match the pixel values to a spatial resolution of quantization parameter values (QP) values in a look up table (LUT). The LUT table values then provide the metadata to provide to the encoder to enable compression of the original picture to be performed by the encoder in a manner so that bits are allocated to pixels in a macroblock according to the predictions of eye tracking.
1. A system for allocating bits for video signal processing and compression, the system comprising:
an input module configured to receive an original picture in a video sequence;
a noise reducer to modify the original picture in a video signal to improve compression efficiency wherein the noise reducer configured to:
receive an original picture from the input module;
obtain a perceptual significance pixel map for the original picture using predictions of eye tracking;
provide an absolute value for numbers in the significance pixel map;
perform companding to reduce the determined absolute value of the pixel values to values ranging from zero to one to produce a weighting map;
apply the weighting map to the original picture to reduce noise in the original picture selectively in a pixel-by-pixel basis to produce a noise-reduced picture;
output the noise-reduced picture;
output the weighting map to a control processor; and
a control processor configured to:
receive the weighting map from the noise reducer;
receive a map of transform-quantization units that specifies size and location of transform-quantization units to be processed by an encoder;
calculate the average value of the weighting map within each transform-quantization unit to produce a decimated weighting map;
determine a quantization parameter for each transform-quantization unit from a quantization parameter (OP) look up table (LUT) that assigns decimated weighting map values to corresponding quantization parameter values;
provide the QP LUT table values as metadata to enable compression of the original picture in a manner so that bits are allocated to pixels in a macroblock according to the predictions of eye tracking; and
an encoder having a first input receiving the original video from the input module, and a control input for receiving the quantization metadata and using the quantization metadata to encode the original video.
2. The system of claim 1 , wherein local quantization strength is adjusted so that the perceptual significance map has pixels within an image identified as perceptually significant receiving preferentially more bits than areas not identified as perceptually significant.
3. The system of claim 1 , wherein the prediction of eye tracking comprises a prediction of visual attention.
4. The system of claim 3 , wherein the prediction of visual attention comprises creating an eye attractor map that models both spatial and temporal aspects of the human vision system.
5. The system of claim 3 , wherein the prediction of visual attention comprises creating an eye attractor map that models either spatial or temporal aspects of the human vision system.
6. The system of claim 5 , wherein the eye attractor map is generated in a perceptual video processor (PVP).
7. The system of claim 6 , wherein the eye attractor map is generated using a spatial temporal function of an adaptive preservation process.
8. The system of claim 6 , wherein the control processor includes the PVP processor to perform the step of obtaining a perceptual significance pixel map that is separate from one or more processors included in the control processor that are used to perform other functions of the control processor.
9. The system of claim 1 , wherein the decimation is performed by converting each block of the companded spatial detail values to a quantization-need value to provide a one-to-one mapping between quantization-need values and QP LUT values.
10. The system of claim 9 , wherein the decimation is performed by averaging values for groups of the pixels.
11. The system of claim 1 , wherein prior to the decimation the control processor is configured to:
perform an input impulse response (IIR) on the companded pixel values to provide a time average gradual change from a previous picture for encoding prior to the pixel values of the current picture.
12. A method for video processing comprising:
receiving an original picture from the input module;
obtaining a perceptual significance pixel map for the original picture using predictions of eye tracking;
providing an absolute value for numbers in the significance pixel map;
performing companding to reduce the determined absolute value of the pixel values to values ranging from zero to one to produce a weighting map;
applying the weighting map to the original picture to reduce noise in the original picture selectively in a pixel-by-pixel basis to produce a noise-reduced picture;
receiving a map of transform-quantization units that specifies size and location of transform-quantization units to be processed by an encoder;
calculating the average value of the weighting map within each transform-quantization unit to produce a decimated weighting map;
determining a quantization parameter for each transform-quantization unit from a quantization parameter (QP) look up table (LUT) that assigns decimated weighting map values to corresponding quantization parameter values;
providing the QP LUT table values as metadata to an encoder to enable the encoder to perform compression of the original picture in a manner so that bits are allocated to pixels in a macroblock according to the predictions of eye tracking.
13. The method of claim 12 , wherein the prediction of visual attention comprises creating an eye attractor map that models either spatial or temporal aspects of the human vision system.
14. The method of claim 12 , further comprising:
performing an input impulse response (IIR) on the companded pixel values to provide a time average gradual change from a previous picture for encoding prior to the pixel values of the current picture.
15. The system of claim 1 , wherein the video processing module comprises a system for adaptively reducing non-noise spatial detail in video to selectively soften features in a picture according to a weighting map derived from significance pixel map.
16. The system of claim 1 , wherein the video processor module comprises an adaptive detail reducer (ADR).
17. The system of claim 1 , wherein the weighting map for the perceptual significance pixel map is transmitted to a remote encoder over a communications network.
18. The system of claim 1 , wherein the perceptual significance pixel map is obtained from predictions of eye tracking from both an original picture and a reference picture.
19. The system of claim 18 , wherein motion compensation is applied to the perceptual significance pixel map for the original picture or the reference picture.