Temporal domain rate distortion optimization considering coding-mode adaptive distortion propagation
A temporal domain rate distortion optimization considering a coding-mode adaptive distortion propagation is provided. A problem of dependency rate distortion optimization based on a temporal domain distortion propagation is induced again according to a temporal domain dependency relationship under an LD structure and a distortion propagation analysis under a skip mode and an inter mode; and an aggregation distortion of a current coding unit and an affected future coding unit are estimated and a propagation factor of a coding unit in a temporal domain distortion propagation model is calculated by constructing a time propagation chain, wherein a Lagrange multiplier is adjusted through an accurate propagation factor to realize a temporal domain dependency rate distortion optimization, and an I frame is subjected to a secondary coding technology to realize the temporal domain dependency rate distortion optimization of the I frame.
1. A temporal domain rate distortion optimization method considering a coding-mode adaptive distortion propagation, comprising the following steps:
S 1 : defining a reconstruction distortion D of a coding unit B i as:
D i =p inter ·D i inter +p skip ·D i skip =d inter +d skip ;
wherein D i inter and D i skip are coding distortions of a current coding unit selecting an inter mode and a skip mode respectively, p inter and p skip are probabilities of the current coding unit selecting the inter mode and the skip mode respectively, d inter is a first partial distortion under the inter mode, d skip is a second partial distortion under the skip mode, p inter +p skip =1;
defining p inter as:
p
inter
=
12
D
i
OMCP
12
D
i
OMCP
+
Δ
2
;
(
1
)
wherein D i OMCP =∥F i −F i−1 ∥ 2 is an original motion compensation error obtained by the coding unit B i in an original frame through a motion search, F i and F i−1 represents original pixels of the coding unit B i and a reference unit B i−1 respectively, and Δ is a quantified step size;
S 2 : when coding the coding unit B i evaluating a partial derivative on a B i temporal domain dependency rate distortion optimization problem
min
o
i
∑
j
=
i
N
E
(
D
j
)
+
λ
g
R
i
(
o
i
)
with respect to R i to obtain a global Lagrange multiplier λ g :
λ
g
=
-
∂
∑
j
=
i
N
E
(
D
j
)
∂
R
i
;
(
2
)
wherein o i is a coding parameter of the coding unit Bi and R i represents a bit number of the coding unit Bi;
multiplying a ∂R i /∂D i at both ends of formula (2) and making ∂D i /∂R i =−λ i to obtain:
λ
i
=
λ
g
/
(
1
+
∂
∑
j
=
i
+
1
N
E
(
D
j
)
∂
D
i
)
=
λ
g
1
+
κ
i
;
(
3
)
wherein λ i is a Lagrange multiplier of the coding unit B i under a global rate distortion performance, and κ i represents an influence of the coding unit B i on a subsequent video sequence coding distortion and is defined as a propagation factor of the coding unit B i ;
S 3 : establishing an aggregation distortion of coding units influenced by the coding unit B i in four coding frames in a current group of pictures (GOP):
∑
k
=
0
3
E
(
D
i
+
k
+
1
)
=
∑
k
=
0
3
(
∑
i
=
0
k
P
i
,
i
+
k
+
1
-
t
·
γ
i
,
i
+
k
+
1
-
t
∏
j
=
i
+
k
+
1
-
t
i
+
k
P
j
,
j
+
1
·
γ
j
,
j
+
1
)
·
D
i
inter
+
L
i
;
(
4
)
wherein γ i,i+k+1−t =α··(p i,i+k+1−t inter ·e −bR i,i+k+1−t +p i,i+k+1−t skip ) α is a constant, P i,i+k+1−t inter and P i,i+k+1−t skip respectively represent probabilities of using the inter mode and the skip mode when a coding unit B i+k+1−t is referenced to the coding unit B i , γ j,j+1 =α··(p j,j+1 inter ·e −bR j+1 +p j,j+1 skip ), wherein P j,j+1 inter and P j,j+1 skip represent probabilities of using the inter mode and the skip mode when the coding unit B j+1 is referenced to the coding unit B j , P i,i+k+1−t represents a probability that the coding frame f i is referenced by the coding frame f i+k+1−t , and P i,i+1 represents a probability that the coding frame f j is referenced by the coding unit f j+1 , and
L
i
=
∑
k
=
0
3
c
i
+
k
+
1
is irrelevant to the coding parameter o i of the coding unit B i , wherein c i+k+1 is an irrelevant item that is irrelevant to the coding parameter o i of the coding unit B i ;
establishing the aggregation distortion of the coding units influenced by the coding unit B i in the four coding frames in an m-th GOP:
∑
k
=
0
3
E
(
D
i
+
4
m
+
k
+
1
)
=
{
∑
k
=
0
3
(
∑
t
=
0
k
P
i
+
4
m
,
i
+
4
m
+
k
+
1
-
t
·
γ
i
+
4
n
,
i
+
4
m
+
k
+
1
-
t
∏
j
=
i
+
4
m
+
k
+
1
-
t
i
+
4
m
+
k
P
j
,
j
+
1
·
γ
j
,
j
+
1
)
·
∏
s
=
0
m
-
1
(
∑
t
=
0
3
P
i
+
4
s
,
i
+
4
s
+
4
-
t
·
γ
i
+
4
s
,
j
+
4
s
+
4
-
t
∏
j
=
i
+
4
s
+
4
-
t
i
+
4
s
+
3
P
j
,
j
+
1
·
γ
j
,
j
+
1
)
}
D
i
inter
+
L
4
m
;
(
5
)
wherein γ i+4m,j+4m+k+1−t =α··(p i+4m,i+4m+k+1−t inter ·e −bR i+4m+k+1−t +p i+4m,i+4m+k+1−t skip ), P i+4m,i+4m+k+1−t inter and P i+4m,i+4m+k+1−t skip respectively represent probabilities of using the inter mode and the skip mode when a coding unit B i+4m+k is referenced to a coding unit B i+4m , P i+4m,i+4m+k+1−t represents a probability that a coding frame f i+4m is referenced by a coding frame f i+4m+k+1−t , P j,j+1 represents a probability that the coding frame f i is referenced by the coding frame f j+1 , and
L
4
m
=
∑
k
=
0
3
c
i
+
4
m
+
k
+
1
is irrelevant to the coding parameter o i of the coding unit B i , wherein c i+4m+k+1 is an irrelevant item irrelevant to the coding parameter o i of the coding unit B i ;
obtaining the aggregation distortion of the coding units affected by the coding unit B i in subsequent coding frames from a coding frame f i+1 to a last coding frame f N :
∑
j
=
i
+
1
N
E
(
D
j
)
=
L
+
∑
m
=
0
M
{
∑
k
=
0
3
(
∑
t
=
0
k
P
i
+
4
m
,
i
+
4
m
+
k
+
1
-
t
·
γ
i
+
4
m
,
i
+
4
m
+
k
+
1
-
t
∏
j
=
i
+
4
m
+
k
+
1
-
t
i
+
4
m
+
k
P
j
,
j
+
1
·
γ
j
,
j
+
1
)
·
∏
s
=
0
m
-
1
(
∑
t
=
0
3
P
i
+
4
s
,
i
+
4
s
+
4
-
t
·
γ
i
+
4
s
,
i
+
4
s
+
4
-
t
∏
j
=
i
+
4
s
+
4
-
t
i
+
4
s
+
3
P
j
,
j
+
1
·
γ
j
,
j
+
1
)
}
D
i
inter
;
(
6
)
wherein M is a total number of a GOP from the coding frame f i+1 to the last coding frame f N , and L represents an item irrelevant to the coding parameter o i ; and
S 4 : according to a definition of p inter in the step S 1 , obtaining a relationship between an inter distortion D i inter of the current coding unit and the reconstruction distortion ID
D
i
inter
=
e
-
bR
i
D
i
1
+
(
e
-
bR
i
-
1
)
p
i
inter
;
(
7
)
wherein b is a constant relevant to an information source distribution, making
e
-
bR
i
1
+
(
e
-
bR
i
-
1
)
p
i
inter
=
η
i
,
and formula (7) is simplified and represented as D i inter =η i D i ;
according to formula (3) in the step S 2 , obtaining a calculation formula of the propagation factor κ i :
κ
i
=
∂
∑
j
=
i
+
1
N
E
(
D
j
)
∂
D
i
=
η
i
∑
m
=
0
M
{
∑
k
=
0
3
(
∑
t
=
0
k
P
i
+
4
m
,
i
+
4
m
+
k
+
1
-
t
·
γ
i
+
4
m
,
i
+
4
m
+
k
+
1
-
t
∏
j
=
i
+
4
m
+
k
+
1
-
t
i
+
4
m
+
k
P
j
,
j
+
1
·
γ
j
,
j
+
1
)
·
∏
s
=
0
m
-
1
(
∑
t
=
0
3
P
i
+
4
s
,
i
+
4
z
+
4
-
t
·
γ
i
+
4
z
,
i
+
4
s
+
4
-
t
∏
j
=
i
+
4
z
+
4
-
t
i
+
4
s
+
3
P
j
,
j
+
1
·
γ
j
,
j
+
1
)
}
;
(
8)
performing an adaptive adjustment on a coding tree unit (CTU)-level global Lagrange multiplier λ g by using the propagation factor κ i , evaluating an average value of the propagation factor κ i for a CTU block according to the above-mentioned steps to obtain the propagation factor κ i of the CTU block, further adjusting a CTU-level QP, and adjusting frame-level QP of B frames by using a frame-level average propagation factor; and adopting a 2-pass coding mode for an I frame, establishing a distortion propagation chain by using a coding distortion obtained at a first coding, calculating the propagation factor κ i of each 16*16 block in the I frame according to the above step, and adjusting the frame-level QP of the I frame by using the frame-level average propagation factor, wherein the QP of the I frame is capable of adjusting influence of a subsequent coding frame according to the I frame.