Mode selection techniques for multimedia coding
This disclosure describes techniques for improving mode selection decisions during the encoding of macroblocks (or other blocks) of multimedia frames of a multimedia sequence. During motion estimation, the encoding modes for macroblocks can be determined so that a desirable encoding rate and acceptable levels of distortion (i.e., acceptable rate-distortion) can be achieved. The techniques may include selecting a set of multimedia coding modes between at least two sets of possible multimedia coding modes for a macroblock of a multimedia frame based on a detail metric associated with the macroblock and mode information associated with neighboring blocks to the macroblock.
1. A method for processing multimedia data comprising:
selecting, by an implementation consisting of a technique in conjunction with hardware, a non-transitory machine readable medium using a processor, or a combination thereof, a set of multimedia coding modes from at least two sets of possible multimedia coding modes for a block of a multimedia frame based on one or more probability equations that include variables corresponding to mode information associated with neighboring blocks to the block, the probability equations including at least one of;
h
0
=
H
0
(
b
)
+
α
q
0
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
OR
h
1
=
H
1
(
b
)
+
α
q
1
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
Wherein q0 and q1 comprise values determined from the mode information associated with the neighboring blocks to the block, b comprises a detail metric, H0(b) comprises a first probability value for the detail metric, H1(b) comprises a second probability value for the detail metric, and α comprises a weighting factor.
2. The method of claim 1 , wherein the block comprises a 16 pixel by 16 pixel macroblock.
3. The method of claim 1 , wherein the multimedia coding modes define granularity of the block in terms of partitions, sub-partitions, or partitions and sub-partitions.
4. The method of claim 1 , wherein selecting from at least two sets of possible multimedia coding modes comprises selecting between a set of inter-planar modes and a set of inter-detail modes.
5. The method of claim 4 , wherein the inter-planar modes comprise an inter 16 by 16 mode, an inter 16 by 8 mode and an inter 8 by 16 mode; and wherein the inter-detail modes comprise an inter 8 by 8 mode, an inter 8 by 4 mode, an inter 4 by 8 mode, and an inter 4 by 4 mode.
6. The method of claim 1 , wherein the multimedia frame is part of a sequence of multimedia frames, each multimedia frame comprising a plurality of blocks, and the method further comprising preprocessing the sequence of multimedia frames, prior to the selecting, to determine a detail metric for each block in the sequence of multimedia frames.
7. The method of claim 1 , further comprising updating statistics in the one or more probability equations for mode selection decisions of subsequent blocks of the frame of the multimedia sequence.
8. The method of claim 1 , wherein the detail metric is indicative of temporal and spatial detail in the block.
9. The method of claim 1 , wherein the mode information associated with the neighboring blocks to the block is defined by a first sub-set of blocks within the frame and a second sub-set of blocks within a different frame.
10. The method of claim 9 , further comprising weighting the first sub-set of blocks within the frame differently than the second sub-set of blocks within the different frame.
11. The method of claim 1 , further comprising selecting a mode for coding the block from the selected set of possible multimedia coding modes.
12. The method of claim 11 , further comprising avoiding motion searches associated with modes for the block in a non-selected set of multimedia coding modes.
13. An apparatus for processing multimedia data comprising:
an encoder that selects a set of multimedia coding modes from at least two sets of possible multimedia coding modes for a block of a multimedia frame based on one or more probability equations that include variables corresponding to mode information associated with neighboring blocks to the block, the probability equations including at least one of;
h
0
=
H
0
(
b
)
+
α
q
0
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
OR
h
1
=
H
1
(
b
)
+
α
q
1
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
Wherein q0 and q1 comprise values determined from the mode information associated with the neighboring blocks to the block, b comprises a detail metric, H0(b) comprises a first probability value for the detail metric, H1(b) comprises a second probability value for the detail metric, and α comprises a weighting factor.
14. The apparatus of claim 13 , wherein the block comprises a 16 pixel by 16 pixel macroblock.
15. The apparatus of claim 13 , wherein the multimedia coding modes define a granularity of the block in terms of partitions, sub-partitions, or partitions and sub-partitions.
16. The apparatus of claim 13 , wherein the encoder selects from at least two sets of possible multimedia coding modes by selecting between a set of inter-planar modes and a set of inter-detail modes.
17. The apparatus of claim 13 , wherein the one or more probability equations comprise the detail metric and the mode information associated with the neighboring blocks as variables.
18. The apparatus of claim 17 , wherein the encoder updates statistics in the one or more probability equations for mode selection decisions of subsequent blocks of the frame of the multimedia sequence.
19. The apparatus of claim 13 , wherein the mode information associated with the neighboring blocks to the block is defined by a first sub-set of blocks within the frame and a second sub-set of blocks within a different frame.
20. The apparatus of claim 19 , wherein the encoder weights the first sub-set of blocks within the frame differently than the second sub-set of blocks within the different frame.
21. The apparatus of claim 13 , wherein the encoder selects a mode for coding the block from the selected set of possible multimedia coding modes.
22. The apparatus of claim 21 , wherein the encoder avoids motion searches associated with modes for the block in a non-selected set of multimedia coding modes.
23. A processor implemented in hardware for processing multimedia data, the processor being configured to:
select a set of multimedia coding modes from at least two sets of possible multimedia coding modes for a block of a multimedia frame based on one or more probability equations that include variables corresponding to mode information associated with neighboring blocks to the block, the probability equations including at least one of;
h
0
=
H
0
(
b
)
+
α
q
0
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
OR
h
1
=
H
1
(
b
)
+
α
q
1
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
Wherein q0 and q1 comprise values determined from the mode information associated with the neighboring blocks to the block, b comprises a detail metric, H0(b) comprises a first probability value for the detail metric, H1(b) comprises a second probability value for the detail metric, and α comprises a weighting factor.
24. The processor of claim 23 , wherein the block comprises a 16 pixel by 16 pixel macroblock.
25. The processor of claim 23 , wherein the multimedia coding modes define a granularity of the block in terms of partitions, sub-partitions, or partitions and sub-partitions.
26. The processor of claim 23 , wherein the processor is configured to select from at least two sets of possible multimedia coding modes by selecting between a set of inter-planar modes and a set of inter-detail modes.
27. The processor of claim 23 , wherein the multimedia frame is part of a sequence of multimedia frames, each multimedia frame comprising a plurality of blocks, and the processor is further configured to preprocess the sequence of multimedia frames, prior to the selecting, to determine a detail metric for each block in the sequence of multimedia frames.
28. The processor of claim 23 , wherein the processor is configured to update statistics in the one or more probability equations for mode selection decisions of subsequent blocks of the frame of the multimedia sequence.
29. A non-transitory machine-readable medium comprising:
instructions for processing multimedia data that upon execution cause a machine to perform a method comprising:
pre-processing a sequence of multimedia frames, each multimedia frame comprising a plurality of blocks; and
selecting a set of multimedia coding modes from at least two sets of possible multimedia coding modes for one of the plurality of blocks of one of the plurality of multimedia frames based on one or more probability equations that include variables corresponding to mode information associated with neighboring blocks to the one of the plurality of blocks, the probability equations including at least one of;
h
0
=
H
0
(
b
)
+
α
q
0
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
OR
h
1
=
H
1
(
b
)
+
α
q
1
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
Wherein q0 and q1 comprise values determined from the mode information associated with the neighboring blocks to the block, b comprises a detail metric, H0(b) comprises a first probability value for the detail metric, H1(b) comprises a second probability value for the detail metric, and α comprises a weighting factor.
30. The machine-readable medium of claim 29 , wherein the multimedia coding modes define a granularity of the block in terms of partitions, sub-partitions, or partitions and sub-partitions.
31. The machine-readable medium of claim 29 , wherein the selecting from the at least two sets of possible multimedia coding modes further comprises selecting between a set of inter-planar modes and a set of inter-detail modes.
32. The machine-readable medium of claim 29 , wherein the one or more probability equations comprise the detail metric and the mode information associated with the neighboring blocks as variables.
33. The machine-readable medium of claim 32 , wherein the method further comprising updating update statistics in the one or more probability equations for mode selection decisions of subsequent blocks of the frame of the multimedia sequence.
34. An apparatus for processing multimedia data comprising:
means for selecting a set of multimedia coding modes from at least two sets of possible multimedia coding modes for a block of a multimedia frame based on one or more probability equations that include variables corresponding to mode information associated with neighboring blocks to the block, the probability equations including at least one of;
h
0
=
H
0
(
b
)
+
α
q
0
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
OR
h
1
=
H
1
(
b
)
+
α
q
1
(
H
0
(
b
)
+
α
q
0
)
+
(
H
1
(
b
)
+
α
q
1
)
,
Wherein q0 and q1 comprise values determined from the mode information associated with the neighboring blocks to the block, b comprises a detail metric, H0(b) comprises a first probability value for the detail metric, H1(b) comprises a second probability value for the detail metric, and α comprises a weighting factor.
35. The apparatus of claim 34 , wherein the multimedia coding modes define a granularity of the block in terms of partitions, sub-partitions, or partitions and sub-partitions.
36. The apparatus of claim 34 , wherein the means selects from at least two sets of possible multimedia coding modes by selecting between a set of inter-planar modes and a set of inter-detail modes.
37. The apparatus of claim 34 , wherein the detail metric is indicative of a number of bits that the block needs in order to be encoded at a constant visual quality.
38. The apparatus of claim 34 , further comprising means for updating statistics in the one or more probability equations for mode selection decisions of subsequent blocks of the frame of the multimedia sequence.
39. The method of claim 1 , wherein the detail metric is indicative of a number of bits that the block needs in order to be encoded at a constant visual quality.
40. The method of claim 1 , wherein the one or more probability equations comprise the detail metric and the mode information associated with the neighboring blocks as variables.
41. The apparatus of claim 13 , wherein the detail metric is indicative of a number of bits that the block needs in order to be encoded at a constant visual quality.
42. The apparatus of claim 13 , wherein the multimedia frame is part of a sequence of multimedia frames, each multimedia frame comprising a plurality of blocks, and the method further comprising preprocessing the sequence of multimedia frames, prior to the selecting, to determine a detail metric for each block in the sequence of multimedia frames.
43. The apparatus of claim 13 , wherein the detail metric is indicative of temporal and spatial detail in the block.
44. The apparatus of claim 16 , wherein the inter-planar modes comprise an inter 16 by 16 mode, an inter 16 by 8 mode and an inter 8 by 16 mode; and wherein the inter-detail modes comprise an inter 8 by 8 mode, an inter 8 by 4 mode, an inter 4 by 8 mode, and an inter 4 by 4 mode.
45. The processor of claim 23 , wherein the detail metric is indicative of a number of bits that the block needs in order to be encoded at a constant visual quality.
46. The processor of claim 26 , wherein the inter-planar modes include an inter 16 by 16 mode, an inter 16 by 8 mode and an inter 8 by 16 mode; and wherein the inter-detail modes include an inter 8 by 8 mode, an inter 8 by 4 mode, an inter 4 by 8 mode, and an inter 4 by 4 mode.
47. The processor of claim 23 , wherein the one or more probability equations comprise the detail metric and the mode information associated with the neighboring blocks as variables.
48. The processor of claim 23 , wherein the detail metric is indicative of temporal and spatial detail in the block.
49. The processor of claim 23 , wherein the mode information associated with the neighboring blocks to the block is defined by a first sub-set of blocks within the frame and a second sub-set of blocks within a different frame.
50. The processor of claim 49 , further comprising weighting the first sub-set of blocks within the frame differently than the second sub-set of blocks within the different frame.
51. The processor of claim 23 , further comprising selecting a mode for coding the block from the selected set of possible multimedia coding modes.
52. The processor of claim 51 , further comprising avoiding motion searches associated with modes for the block in a non-selected set of multimedia coding modes.
53. The machine-readable medium of claim 29 , wherein the block comprises a 16 pixel by 16 pixel macroblock.
54. The machine-readable medium of claim 29 , wherein the detail metric is indicative of a number of bits that the block needs in order to be encoded at a constant visual quality.
55. The machine-readable medium of claim 31 , wherein the inter-planar modes include an inter 16 by 16 mode, an inter 16 by 8 mode and an inter 8 by 16 mode; and wherein the inter-detail modes include an inter 8 by 8 mode, an inter 8 by 4 mode, an inter 4 by 8 mode, and an inter 4 by 4 mode.
56. The machine-readable medium of claim 29 , wherein the detail metric is indicative of temporal and spatial detail in the block.
57. The machine-readable medium of claim 29 , wherein the mode information associated with the neighboring blocks to the block is defined by a first sub-set of blocks within the frame and a second sub-set of blocks within a different frame.
58. The machine-readable medium of claim 57 , further comprising weighting the first sub-set of blocks within the frame differently than the second sub-set of blocks within the different frame.
59. The machine-readable medium of claim 29 , further comprising selecting a mode for coding the block from the selected set of possible multimedia coding modes.
60. The machine-readable medium of claim 59 , further comprising avoiding motion searches associated with modes for the block in a non-selected set of multimedia coding modes.
61. The machine-readable medium of claim 29 , wherein the multimedia frame is part of a sequence of multimedia frames, each multimedia frame comprising a plurality of blocks, and the preprocessing the sequence of multimedia frames further comprises preprocessing the sequence of multimedia frames, prior to the selecting, to determine a detail metric for each block in the sequence of multimedia frames.
62. The apparatus of claim 34 , wherein the block comprises a 16 pixel by 16pixel macroblock.
63. The apparatus of claim 36 , wherein the inter-planar modes include an inter 16 by 16 mode, an inter 16 by 8 mode and an inter 8 by 16 mode; and wherein the inter-detail modes include an inter 8 by 8 mode, an inter 8 by 4 mode, an inter 4 by 8 mode, and an inter 4 by 4 mode.
64. The apparatus of claim 34 , wherein the detail metric is indicative of temporal and spatial detail in the block.
65. The apparatus of claim 34 , wherein the mode information associated with the neighboring blocks to the block is defined by a first sub-set of blocks within the frame and a second sub-set of blocks within a different frame.
66. The apparatus of claim 65 , further comprising weighting the first sub-set of blocks within the frame differently than the second sub-set of blocks within the different frame.
67. The apparatus of claim 34 , further comprising selecting a mode for coding the block from the selected set of possible multimedia coding modes.
68. The apparatus of claim 67 , further comprising avoiding motion searches associated with modes for the block in a non-selected set of multimedia coding modes.
69. The apparatus of claim 34 , wherein the multimedia frame is part of a sequence of multimedia frames, each multimedia frame comprising a plurality of blocks, and the method further comprising preprocessing the sequence of multimedia frames, prior to the selecting, to determine a detail metric for each block in the sequence of multimedia frames.
70. The apparatus of claim 34 , wherein the one or more probability equations comprise the detail metric and the mode information associated with the neighboring blocks as variables.