Video sequence encoding system and algorithms
View Patent ↗The video sequence encoding system with high compression efficiency is based on the H.264/AVC international video coding standard and is implementing proposed algorithms for: macroblock motion estimation; simplified rate-distortion optimization for transform size decision-making; acceleration of optimal macroblock type decision; modulation of quantization parameter with look-ahead refinement; and film-grain parameters calculation. The video encoding system comprises a motion estimation unit based on a measurement function for motion search of the matching texture block; a macroblock decision making unit; a pipeline processing module configured to accelerate complex transform; and a quantization parameter modulator configured to enhance the picture quality.
1. A video codec with high compression efficiency, the video codec comprising:
a macroblock encoding module configured to receive a digitized picture divided into macroblocks, encode at least some of the macroblocks of the picture and output a compressed stream of pixels; and
a quantization parameter modulator operably coupled to the macroblock encoding module and configured to calculate quantization parameters for the macroblocks;
wherein the macroblock encoding module comprises:
a motion estimation unit configured to estimate macroblock motion based at least in part on a measurement function for motion search of a matching texture block; and
a macroblock decision-making unit that is responsible to the motion estimation unit and that is configured to optimize selection of macroblock type and transform size;
the measurement function for motion search is calculated as:
W ( I,R,M,N ,QP, mvx,mvy )= C ( I )·(SAD( I,R )+Cost( M,N,mvx,mvy ,QP))+ M·N·C ( R ),
where:
M, N—width and height of a current texture block;
I—two-dimensional M×N array of initial texture luminance component values corresponding to the current texture block with elements I(x, y),
where xε[0,M−1], yε[0,N−1];
R—two-dimensional M×N array of texture prediction luminance component values corresponding to a reference texture block with elements R(x, y),
where xε[0,M−1], yε[0,N−1];
QP—a quantization parameter used for video encoding;
mvx,mvy—components of difference between a current motion offset and a current motion vector prediction;
SAD(I, R)—is a Sum of Absolute Differences between corresponding elements of the arrays I and R;
Cost(M,N,mvx,mvy,QP)—a motion vector cost function; and
C(A)—a complexity function for two-dimensional array A, A corresponding to arrays I or R, respectively:
C
(
A
)
=
∑
y
=
0
N
-
1
∑
x
=
0
M
-
1
F
(
A
,
x
,
y
,
B
H
(
x
)
,
B
V
(
y
)
)
;
where
F(A, x, y, dx, dy) is auxiliary function for a two-dimensional M×N array A; and
B H (x), B V (y)—boundary indicator functions for first and second indices of the two-dimensional M×N array, which are equal to:
+1, if index equals a minimal value (zero);
−1, if the index equals a maximal value ((M−1) for B H (x) and (N−1) for B V (y));
and 0—otherwise.
2. The video codec according to claim 1 , further comprising a pre-processing module configured to perform input video preliminary processing, including:
dividing the digitized picture into groups of pixels to thereby provide the macroblocks;
performing format transforms;
performing filtering; and
providing quality improvement.
3. The video codec according to claim 2 , wherein the pre-processing module is configured to provide pipelined video processing.
4. The video codec according to claim 1 , further comprising a film grain calculation module configured to restore an original film-grain effect.
5. A method of video encoding using a codec with high compression efficiency, the method comprising:
dividing an input picture into macroblocks of pixels;
calculating quantization parameters for at least some of the macroblocks;
performing motion estimation based on a measurement function for motion search of a matching texture block;
calculating an optimal macroblock type; and
selecting an optimal transform size for corresponding macroblock transforms;
wherein the measurement function for motion estimation is calculated as:
W ( I,R,M,N ,QP, mvx,mvy )= C ( I )·(SAD( I,R )+Cost( M,N,mvx,mvy ,QP))+ M·N·C ( R ),
where:
M, N—width and height of a current texture block;
I—two-dimensional M×N array of initial texture luminance component values corresponding to the current texture block with elements I(x, y), where xε[0,M−1], yε[0,N−1];
R—two-dimensional M×N array of texture prediction luminance component values corresponding to a reference texture block with elements R(x, y), where xε[0,M−1], yε[0,N−1];
QP—a quantization parameter used for video encoding;
mvx,mvy—components of difference between a current motion offset and a current motion vector prediction;
SAD(I,R)—is a Sum of Absolute Differences between corresponding elements of the arrays I and R;
Cost(M,N,mvx,mvy,QP)—a motion vector cost function; and
C(A)—a complexity function for two-dimensional array A, A corresponding to arrays I or R, respectively:
C
(
A
)
=
∑
y
=
0
N
-
1
∑
x
=
0
M
-
1
F
(
A
,
x
,
y
,
B
H
(
x
)
,
B
V
(
y
)
)
;
where
F(A, x, y, dx, dy) is auxiliary function for a two-dimensional M×N array A; and
B H (x), B V (y)—boundary indicator functions for first and second indices of the two-dimensional M×N array, which are equal to:
+1, if index equals a minimal value (zero);
−1, if the index equals a maximal value ((M−1) for B H (x) and (N−1) for B V (y));
and 0—otherwise.
6. The method of video encoding according to claim 5 , comprising preliminary processing input video using pipelined video processing.
7. The method of pipelined video processing in a video encoding system of claim 6 , comprising:
providing at least one elementary object, the object having one input data pipe and one output data pipe;
sending at least one line of pixels of an input image from an external memory to the input data pipe;
storing input pixel lines in an internal object memory;
when the number of input lines in the internal object memory equals a predefined number, calculating a line of output pixels;
forwarding at least one line of output pixels to the output data pipe;
repeating until processing of the input image is complete; and
combining lines of output pixels into an output image.
8. The method of claim 7 , wherein, in case of complex transform, the method comprises:
providing a plurality of elementary objects corresponding to elementary transforms;
combining the plurality of elementary objects into a chain having one input data pipe and one output data pipe, each object being responsible for a respective transform;
sending the input image pixels into input data pipe of the chain; and
receiving the lines of output pixels from the chain output data pipe.
9. The method of video encoding according to claim 5 , wherein the calculation of quantization parameters for at least some of the macroblocks is performed using a look-ahead approach.
10. The method of calculating quantization parameters in a video sequence encoder of claim 9 , comprising:
creating a reduced resolution picture for each frame in the video sequence;
performing block-based motion estimation for each reduced resolution picture beginning with a second frame of the video sequence;
calculating Inter-complexity and Intra-complexity functions for each macroblock in the reduced resolution picture;
calculating importance as a minimal value of the Inter-complexity function and the Intra-complexity function;
creating an importance map;
filtering the importance map using a multi-pass non-linear filter; and
modulating a quantization parameter using the importance map.
11. The method of modulating the quantization parameter of claim 10 , comprising reducing direct resolution twice in each direction and calculating each pixel in the reduced resolution picture as:
RRP[ x][y ]=(Luma_Orig[ x* 2 ][y* 2]+Luma_Orig[ x* 2+1 ][y* 2]+Luma_Orig[ x* 2 ][y* 2+1]++Luma_Orig[ x* 2+1 ][y* 2+1]+2)/4.
12. The method of claim 10 , comprising refining an importance map of a previous reduced resolution picture using motion information from a current reduced resolution picture.
13. The method of claim 12 , wherein an importance is calculated as:
NewImportance[ x+mvx][y+mvy ]=(NewImportance[ x+mvx][y+mvy ]+CurrentImportance[ x][y ])/4,
where:
mvx and mvy—motion vectors found during block-based motion estimation for the current picture.
14. The method of claim 5 , wherein the auxiliary function F(A, x, y, dx, dy) is calculated as:
F
=
1
8
·
9
·
A
(
x
,
y
)
-
∑
k
=
-
1
1
∑
j
=
-
1
1
A
(
x
+
j
,
y
+
k
)
,
if
dx
=
0
,
dy
=
0
;
F
=
1
4
4
·
A
(
x
,
y
)
-
A
(
x
+
dx
,
y
)
-
A
(
x
,
y
+
dy
)
-
2
·
A
(
x
+
dx
,
y
+
dy
)
if
dx
≠
0
,
dy
≠
0
;
F
=
1
8
·
8
·
A
(
x
,
y
)
-
A
(
x
,
y
-
1
)
-
A
(
x
,
y
+
1
)
-
2
·
(
A
(
x
+
dx
,
y
-
1
)
+
A
(
x
+
dx
,
y
)
+
A
(
x
+
dx
,
y
+
1
)
)
if
dx
≠
0
and
dy
=
0
;
and
F
=
1
8
·
8
·
A
(
x
,
y
)
-
A
(
x
-
1
,
y
)
-
A
(
x
+
1
,
y
)
-
2
·
(
A
(
x
-
1
,
y
+
dy
)
+
A
(
x
,
y
+
dy
)
+
A
(
x
+
1
,
y
+
dy
)
)
if
dx
=
0
and
dy
≠
0.
15. The method of claim 5 , wherein the step of selecting the optimal transform size for a current macroblock transform comprises:
defining a function for quantization of Discrete Cosine (DC) transform coefficients for a 4×4 transform;
defining a function for quantization of Discrete Cosine (DC) transform coefficients for an 8×8 transform;
defining a function for inverse quantization of Discrete Cosine (DC) transform coefficients for a 4×4 transform;
defining a function for inverse quantization of Discrete Cosine (DC) transform coefficients for an 8×8 transform;
calculating a cost function for a 4×4 transform based on the functions for quantization and inverse quantization of DC transform coefficients for a 4×4 transform and a quantization parameter for the current macroblock;
calculating a cost function for an 8×8 transform based on the functions for quantization and inverse quantization of DC transform coefficients for an 8×8 transform and the quantization parameter for the current macroblock;
comparing values of cost functions for a 4×4 transform and for an 8×8 transform; and
selecting an optimal transform size based on the cost function with a minimum value.
16. The method of claim 15 , wherein the cost function for the 4×4 transform is calculated as:
COST
4
×
4
=
∑
j
=
0
15
(
[
DEQUANT
4
×
4
(
QUANT
4
×
4
(
A
(
j
)
)
)
/
4
-
A
(
j
)
]
2
16
)
+
λ
·
∑
j
=
0
15
R
(
QUANT
4
×
4
(
A
(
j
)
)
)
and the cost function for the 8×8 transform is calculated as:
COST
8
×
8
=
∑
j
=
0
15
(
[
DEQUANT
8
×
8
(
QUANT
8
×
8
(
C
(
j
)
)
)
-
C
(
j
)
]
2
64
)
+
λ
·
∑
j
=
0
15
R
(
QUANT
8
×
8
(
C
(
j
)
)
)
where:
QUANT 4×4 (X) is a function for quantization of DC coefficient X of 4×4 transform;
QUANT 8×8 (X) is a function for quantization of DC coefficient X of 8×8 transform;
DEQUANT 4×4 (X) is a function for inverse quantization of DC coefficient X of 4×4 transform;
DEQUANT 8×8 (X) is a function for inverse quantization of DC coefficient X of 8×8 transform;
R
(
x
)
=
{
2
*
x
,
if
x
<
14
,
28
,
if
x
≥
14
;
λ is a Lagrangian multiplier;
A is a one-dimensional array of 16 elements:
A
(
x
+
4
y
)
=
∑
k
=
0
3
∑
j
=
0
3
(
I
(
4
x
+
j
,
4
y
+
k
)
-
P
(
4
x
+
j
,
4
y
+
k
)
)
;
for
x
,
y
∈
[
0
,
3
]
;
I(x, y) is a two-dimensional 16×16 array of initial texture luma component values corresponding to a current block with elements x and y;
P(x, y) is a two dimensional 16×16 array of texture prediction luma component values corresponding to the current block with elements x and y; and
C is a one-dimensional array of 16 elements calculated as:
C
(
j
)
=
{
B
(
j
)
+
B
(
j
+
4
)
,
if
j
∈
[
0
,
3
]
B
(
j
+
4
)
+
B
(
j
+
8
)
,
if
j
∈
[
4
,
7
]
B
(
j
-
8
)
-
B
(
j
-
4
)
,
if
j
∈
[
8
,
11
]
B
(
j
-
4
)
-
B
(
j
)
,
if
j
∈
[
12
,
15
]
wherein values B(j) are defined as:
B (2 k )= A (2 k )+ A (2 k+ 1),
B (2 k+ 1)= A (2 k )− A (2 k+ 1), for kε[ 0;7].
17. A method of accelerating selection of optimal macroblock type in a video sequence encoder with high compression efficiency, comprising:
defining a two-dimensional 16×16 array of picture luma component values corresponding to a current macroblock MB;
choosing a best mode of Intra 16×16 prediction;
choosing best motion vectors and reference frames of Inter prediction;
calculating two-dimensional 16×16 arrays of texture prediction for luma component values corresponding to current macroblock for Intra 16×16 macroblock type P INTRA — 16×16 and for Inter macroblock type P INTER ;
calculating two dimensional arrays of differences R INTRA — 16×16 between MB and P INTRA — 16×16 ;
calculating two dimensional arrays of differences R INTER between MB and P INTER ;
calculating a sum of absolute values SAD(R INTER );
calculating a sum of absolute values SAD(R INTRA — 16×16 );
calculating an auxiliary complexity function C(MB) as:
C
(
MB
)
=
∑
k
=
0
N
2
-
1
∑
j
=
0
N
2
-
1
MB
(
2
j
+
1
,
2
k
+
1
)
-
MB
(
2
j
,
2
k
+
1
)
+
MB
(
2
j
,
2
k
)
+
MB
(
2
j
+
1
,
2
k
)
+
MB
(
min
(
N
-
1
,
2
j
+
2
)
,
2
k
)
4
where
MB(x,y)—elements of 16×16 two-dimensional macroblock texture array with horizontal coordinate x and vertical coordinate y;
N=16; and
min (a,b) is the minimum value of a and b;
wherein no testing of Intra 4×4 macroblock type is required, when the following conditions are satisfied:
SAD( R INTER )< M (QP)·log 2 ( C ( MB )); and
SAD( R INTRA — 16×16 )>max( K ·SAD( R INTER )+ S (QP), T (QP)),
where:
QP is a quantization parameter for the current macroblock; and
K, M(QP), S(QP), T(QP) are parameters selected based on the encoder compression efficiency and acceleration of decision-making procedure.
18. The method of claim 17 , wherein K=5.86, and values of M(QP), S(QP), and T(QP) are each selected from a corresponding array as follows:
M (QP)=
{12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 13.13, 13.48, 13.87, 14.96, 15.66, 16.80, 17.30, 18.36, 19.73, 21.99, 23.67, 26.09, 28.52, 31.84, 34.77, 37.99, 41.02, 44.14, 46.37, 49.10, 52.73, 54.96, 59.61, 63.48, 68.36, 73.24, 78.13, 83.59, 89.06, 94.53, 100.00, 105.47, 114.26, 123.05, 131.84, 140.63, 153.13, 165.63, 178.13, 190.63, 203.13},
S(QP)=
{900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 920, 950, 1000, 1080, 1190, 1350, 1536, 1690, 1870, 2080, 2340, 2600, 2840, 3060, 3280, 3500, 3840, 4230, 4580, 5020, 5500, 6170, 6920, 7620, 8500, 9390, 10310, 11280, 12110, 13000},
T (QP)=
{256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 265, 273, 281, 289, 300, 313, 320, 329, 353, 369, 400, 425, 433, 457, 465, 500, 521, 545, 553, 568, 594, 610, 634, 658, 700, 778, 882, 1000, 1176, 1404, 1653, 1901, 2142, 2400}.