Speedup techniques for rate distortion optimized quantization

1. A method of selecting a coding mode for an image coding unit to be coded and quantized in an image coding process, comprising:

a) performing a first quantization process having a first type of quantization technique for each of M possible coding modes, wherein performing the first quantization process includes quantizing a prediction residual using a quantization parameter;

(b) based on the first quantization process, selecting M 1 best coding modes according to a predefined coding criterion, wherein M>M 1 >1;

(c) after steps (a) and (b) have been performed for each of the M possible coding modes, performing a second quantization process that has a second type of quantization technique for the M 1 best coding modes, wherein performing the second quantization process includes re-quantizing the prediction residual using the quantization parameter; and

(d) selecting a best coding mode based on the results of the first quantization process and the second quantization process;

wherein the first quantization process has the first type of quantization technique, wherein the first type of quantization technique comprises a faster type of quantization than the second quantization type;

wherein the second quantization process has the second type of quantization technique, which comprises a slower quantization type than the first quantization type; and

wherein the first quantization process and the second quantization process are repeated for each quantization parameter.

2. The method as recited in claim 1 , wherein information about the positions and magnitudes of quantized transform coefficients of one or more quantization parameters is used during the quantization processes of subsequent quantization parameters.

3. The method as recited in claim 1 , wherein the second quantization process is applied to a single best coding mode among the M possible coding modes, or equivalently, M 1 =1.

4. The method as recited in claim 1 , wherein the best result is selected based on steps that comprise:

comparing i) a value of the predefined coding criterion of a residual, which is quantized with the first quantization process, with ii) a value of the predefined coding criterion of a residual quantized with the second quantization process; and

selecting the residual that offer a better value of the predefined coding criterion.

5. The method as recited in claim 1 , further comprising re-encoding the image coding unit after the optimal coding mode has been selected after the initial encoding process.

6. The method as recited in claim 5 , wherein the second quantization process is performed while re-encoding of the image coding unit.

7. The method as recited in claim 6 , wherein re-encoding comprises evaluating optimal prediction modes.

8. The method as recited in claim 7 , wherein a subset of prediction modes are evaluated during the re-encoding.

9. The method as recited in claim 8 , wherein the subset of prediction modes that is evaluated during the re-encoding process is selected based on the optimal prediction modes, which are chosen after the initial encoding process.

10. A system for selecting a coding mode for an image coding unit to be coded and quantized in an image coding process, the system comprising:

(a) means for performing a first quantization process having a first type of quantization technique for each of M possible coding modes, wherein performing the first quantization process includes quantizing a prediction residual using a quantization parameter;

(b) means for selecting, based on the first quantization process, M 1 best coding modes according to a predefined coding criterion, wherein M>M 1 >1;

(c) means for performing, after a function of means (a) for performing and a function of means (b) for selecting have been performed for each of the M possible coding modes, a second quantization process that has a second type of quantization technique for the M 1 best coding modes, wherein performing the second quantization process includes re-quantizing the prediction residual using the quantization parameter; and

(d) means for selecting a best coding mode based on the results of the first quantization process and the second quantization process; wherein the first quantization process has the first type of quantization technique,

wherein the first type of quantization technique comprises a faster type of quantization than the second quantization type;

wherein the second quantization process has the second type of quantization technique, which comprises a slower quantization type than the first quantization type; and

wherein the first quantization process and the second quantization process are repeated for each quantization parameter.

11. The system as recited in claim 10 wherein the system comprises one or more of: means for performing or executing a computational function; or means for performing or executing a function that relates to video encoding.

12. An apparatus, comprising: a processor; and a non-transitory computer readable storage medium that comprises instructions, which when performed or executed by the processor, cause, control, program or configure the processor to perform a process for selecting a coding mode for an image coding unit to be coded and quantized in an image coding process, the process comprising:

(a) performing a first quantization process having a first type of quantization technique during evaluation for each of M possible coding modes, wherein performing the first quantization process includes quantizing a prediction residual using a quantization parameter;

(b) based on the first quantization process, selecting M 1 best coding modes according to a predefined coding criterion, wherein M>M 1 >1;

(c) after steps (a) and (b) have been performed for each of the M possible coding modes, performing a second quantization process that has a second type of quantization technique for the M 1 best coding modes, wherein performing the second quantization process includes re-quantizing the prediction residual using the quantization parameter; and

(d) selecting a best coding mode based on the results of the first quantization process and the second quantization process;

wherein the first quantization process has the first type of quantization technique,

wherein the first type of quantization technique comprises a faster type of quantization than the second quantization type;

wherein the second quantization process has the second type of quantization technique, which comprises a slower quantization type than the first quantization type; and

wherein the first quantization process and the second quantization process are repeated for each quantization parameter.

13. The apparatus as recited in claim 11 wherein the apparatus comprises one or more of a computer or a video encoder.

14. A non-transitory computer readable storage medium that comprises instructions encoded or stored therewith, which when performed or executed by a processor, cause, control, program or configure the processor to perform, execute or control a process for selecting a coding mode for an image coding unit to be coded and quantized in image coding, the process comprising:

(a) performing a first quantization process having a first type of quantization technique for each of M possible coding modes wherein performing the first quantization process includes quantizing a prediction residual using a quantization parameter;

(b) based on the first quantization process, selecting M 1 best coding modes according to a predefined coding criterion, wherein M>M 1 >1;

(c) after steps (a) and (b) have been performed for each of the M possible coding modes, performing a second quantization process that has a second type of quantization technique for the M 1 best coding modes, wherein performing the second quantization process includes re-quantizing the prediction residual using the quantization parameter; and

(d) selecting a best coding mode based on the results of the first quantization process and the second quantization process;

wherein the first quantization process has the first type of quantization technique,

wherein the first type of quantization technique comprises a faster type of quantization than the second quantization type;

wherein the second quantization process has the second type of quantization technique, which comprises a slower quantization type than the first quantization type; and

wherein the first quantization process and the second quantization process are repeated for each quantization parameter.