Methods and apparatus for quantization and dequantization of a rectangular block of coefficients

1. A method for performing dequantization on a rectangular block of quantized transform coefficients, the method comprising:

receiving a W×H block of quantized transform coefficients, wherein W is a row width of the block and H is a column height of the block;

applying dequantization processing to the W×H block of quantized transform coefficients to generate a block of transform coefficients, which is used to generate a picture corresponding to the W×H block of quantized transform coefficients, wherein the dequantization processing comprises applying a first dequantization algorithm when an output M of a function M=log 2 (√{square root over (W·H)}) is equal to an integer value, and applying a second dequantization algorithm when the output M is equal to a non-integer value.

2. The method of claim 1 , wherein applying the first dequantization algorithm comprises using a first mapping table to calculate a first set of constants used to generate the block of transform coefficients, and applying the second dequantization algorithm comprises using a second mapping table to calculate a second set of constants used to generate the block of transform coefficients.

3. The method of claim 1 wherein applying the first dequantization algorithm comprises using a first mapping table to calculate a first set of constants used to generate the block of transform coefficients, and applying the second dequantization algorithm comprises using a shifted first mapping table to calculate a second set of constants used to generate the block of transform coefficients.

4. The method of claim 1 , wherein the second dequantization algorithm is applied when W×H=2X 2 .

5. The method of claim 2 , wherein the block of transform coefficients is generated using dequantization function, coeffQ=((level×IQ<<(QP/6))+offset)>>IQbits, wherein:

level is a quantized transform coefficient, from the W×H block of quantized transform coefficients, which is input into the dequantization function;

coeffQ is a corresponding transform coefficient that is output from the dequantization function;

IQ is constant obtained from the first mapping table when the output M is equal to the integer value, otherwise IQ is obtained from the second mapping table;

QP is a decoded quantization parameter;

IQbits is a calculable constant comprising a number of bits used in performing a right shift within the dequantization function;

offset is a calculable constant derived from IQBits.

6. The method of claim 5 , wherein:

the first mapping table comprises IQ=g(QP %6), wherein g(x=0,1,2,3,4,5)={40,45,51,57,64,72}; and

the shifted first mapping table comprises IQ=g((QP+3)%6), wherein g(x=0,1,2,3,4,5)={40,45,51,57,64,72}, and wherein QP is a decoded quantization parameter.

7. The method of claim 3 , wherein:

the first mapping table comprises IQ=g1(QP %6), wherein g1(x=0,1,2,3,4,5)={40,45,51,57,64,72}; and

the second mapping table comprises IQ=g2(QP %6), wherein g2(x=0,1,2,3,4,5)={56,63,72,80,90,101}.

8. A method for performing quantization on a rectangular block of transform coefficients, the method comprising:

receiving a W×H block of transform coefficients generated from a picture, wherein W is a row width of the block and H is a column height of the block;

applying quantization processing to the W×H block of transform coefficients to generate a block of quantized transform coefficients used to generate a bit sequence, which is used to reconstruct the picture, wherein the quantization processing comprises applying a first quantization algorithm when an output M of a function M=log 2 (√{square root over (W·H)}) is equal to an integer value, and applying a second quantization algorithm when the output M is equal to a non-integer value.

9. The method of claim 8 , wherein applying the first quantization algorithm comprises using a first mapping table to calculate a first set of constants used to generate the block of quantized transform coefficients, and applying the second quantization algorithm comprises using a second mapping table to calculate a second set of constants used to generate the block of quantized transform coefficients.

10. The method of claim 9 , wherein the block of quantized transform coefficients is generated using quantization function, level=(coeff×Q+offset)>>Qbits, wherein:

coeff is a transform coefficient, from the W×H block transform coefficients, which is input into the quantization function;

level is a corresponding quantized transform coefficient that is output from the quantization function;

Q is a constant obtained from the first mapping table when the output M is equal to the integer value, otherwise Q is obtained from the second mapping table;

Qbits is a calculable constant comprising a number of bits used in performing a right shift that represents a division operation within the quantization function;

offset is a calculable constant that is derived using Qbits.

11. The method of claim 10 , wherein:

the first mapping table comprises Q1=f1(QP %6), wherein f1(x=0,1,2,3,4,5)={26214,23302,20560,18396,16384,14564}; and

the second mapping table comprises Q2=f2(QP %6), wherein f2(x=0,1,2,3,4,5)={18536,16477,14538,13007,11585,10298}, and wherein QP is a predetermined quantization parameter.

12. The method of claim 10 further comprising determining whether to perform entropy coding of the quantized transform coefficients using the first set of constants, when the output M is equal to the integer value, otherwise determining whether to perform entropy coding of the quantized transform coefficients using the second set of constants.

13. The method of claim 8 , wherein applying the first quantization algorithm comprises using a first mapping table to calculate a first set of constants used to generate the block of quantized transform coefficients, and applying the second quantization algorithm comprises using a shifted first mapping table to calculate a second set of constants used to generate the block of quantized transform coefficients.

14. The method of claim 13 , wherein:

the first mapping table comprises Q1=f(QP %6), wherein f(x=0,1,2,3,4,5)={26214,23302,20560,18396,16384,14564}; and

the shifted first mapping table comprises Q2=f ((QP+3)%6), wherein f(x=0,1,2,3,4,5)={26214,23302,20560,18396,16384,14564}, and wherein QP is a selected quantization parameter.

15. The method of claim 8 , wherein the second quantization algorithm is applied when W×H=2X 2 .

16. A system comprising:

a decoder having a dequantizer block that is configured to:

receive a first W×H block of quantized transform coefficients, wherein W is a row width of the block and H is a column height of the block;

apply dequantization processing to the first block of quantized transform coefficients to generate a first block of transform coefficients, which is used to generate a first picture corresponding to the first W×H block of quantized transform coefficients, wherein the dequantization processing comprises applying a first dequantization algorithm when an output M of a function M=log 2 (√{square root over (W·H)}) is equal to an integer value, and applying a second dequantization algorithm when the output M is equal to a non-integer value.

17. The system of claim 16 , wherein the dequantizer block is further configured to:

calculate a first set of constants using a first mapping table, while applying the first dequantization algorithm, wherein the first set of constants is used within a dequantization function to generate the first block of transform coefficients from the first block of quantized transform coefficients;

calculate a second set of constants using a second mapping table, while applying the second dequantization algorithm, wherein the second set of constants is used within the dequantization function to generate the first block of transform coefficients from the first block of quantized transform coefficients.

18. The system of claim 16 , wherein the dequantizer block is further configured to:

calculate a first set of constants using a first mapping table, while applying the first dequantization algorithm, wherein the first set of constants is used within a dequantization function to generate the first block of transform coefficients from the first block of quantized transform coefficients;

calculate a second set of constants using a shifted first mapping table, while applying the second dequantization algorithm, wherein the second set of constants is used within the dequantization function to generate the first block of transform coefficients from the first block of quantized transform coefficients.

19. The system of claim 16 further comprising:

an encoder having a quantizer block that is configured to:

receive a second W×H block of transform coefficients generated from a second picture, wherein W is a row width of the second block of transform coefficients and H is a column height of the second block of transform coefficients;

apply quantization processing to the second block of transform coefficients to generate a second block of quantized transform coefficients used to generate a bit sequence, which is used to reconstruct the second picture, wherein the quantization processing comprises applying a first quantization algorithm when the output M of the function M=log 2 (√{square root over (W·H)}) is equal to the integer value, and applying a second quantization algorithm when the output M is equal to the non-integer value.

20. The system of claim 16 , wherein the decoder is configured to operate in conformance with a High Efficiency Video Coding Standard.