Apparatus and method for local video detector for mixed cadence sequence
Aspects of the invention are directed towards an apparatus and method for detecting local video pixels in mixed cadence video. The local video detector comprises a comb detector that is adaptive to the contour of moving objects and local contrast, a motion detector that is robust to false motion due to vertical details, and a fader value estimator that provides a video confidence value to a fader that combines film mode and video mode processing results. The coupling of the local video detector to a film mode detector increases the robustness, accuracy, and efficiency of local film/video mode processing as compared to the prior art.
1. A local video detection apparatus, comprising:
a contour and contrast adjusted comb detector;
a false motion excluded motion detector; and
a video fader value estimator coupled to the contour and contrast adjusted comb detector and the false motion excluded motion detector.
2. The apparatus of claim 1 , wherein the contour and contrast adjusted comb detector comprises:
a field to frame coupling unit;
a directional support pixel extraction unit coupled to the field to frame coupling unit;
a comb detection unit coupled to the directional support pixel extraction unit; and
a comb value estimation unit coupled to the directional support pixel extraction unit.
3. The apparatus of claim 2 , further comprising a comb post-processing unit coupled to the comb detection unit.
4. The apparatus of claim 2 , further comprising a contrast adjustment unit coupled to the directional support pixel extraction unit and the comb value estimation unit.
5. The apparatus of claim 2 , wherein the field to frame coupling unit is configured to receive input video signals at times t−1, t, and t+1 and merges a current field of the input video signal at time t with a coupling field of the input video signals at time t−1 or t+1, wherein the current field and the coupling fields belong to a same frame according to a cadence and phase information received from a film mode detector.
6. The apparatus of claim 2 , wherein
the directional support pixel extraction unit is coupled to a contour detection unit and is configured to extract pixels from a frame based on an estimated contour angle provided by a contour detection unit.
7. The apparatus of claim 2 , wherein
the comb detection unit is configured to calculate a comb signal of a pixel based on a vertical frequency according to the equation
vfreq
k
=
{
freq
k
,
1
-
freq
k
,
2
fielddiff
k
≥
MonotoneTh
0
otherwise
k
=
0
,
1
,
2
where
freq
k
,
1
=
∑
i
=
0
3
(
(
y
k
+
i
-
mean
k
)
·
(
y
k
+
i
+
1
-
mean
k
)
<
0
)
freq
k
,
2
=
∑
i
=
0
2
(
(
y
k
+
i
-
mean
k
)
·
(
y
k
+
i
+
2
-
mean
k
)
<
0
)
mean
k
=
(
(
y
k
+
y
k
+
2
+
y
k
+
4
)
⨯
2
+
(
y
k
+
1
+
y
k
+
3
)
⨯
3
+
6
)
/
12
and
fielddiff
k
=
(
y
k
+
y
k
+
2
+
y
k
+
4
)
⨯
2
-
(
y
k
+
1
+
y
k
+
3
)
⨯
3
/
12.
8. The apparatus of claim 7 , wherein the comb signal of the pixel takes into account comb pixels of a neighboring field.
9. The apparatus of claim 8 , wherein the comb signal of the pixel is according to the equation
C
=
{
max
(
vfreq
0
,
vfreq
1
)
topfield
max
(
vfreq
1
,
vfreq
2
)
bottomfield
.
10. The apparatus of claim 7 , wherein the comb signal of the pixel excludes stationary comb pixels.
11. The apparatus of claim 10 , wherein
the comb detection unit is coupled to a motion detection unit providing a recursive motion value to the comb detection unit; and wherein
the comb signal of the pixel is according to the equation
C
′
=
{
C
m
≥
MotionTh
0
otherwise
where
m
is
the
recursive
motion
where
C
=
{
max
(
vfreq
0
,
vfreq
1
)
topfield
max
(
vfreq
1
,
vfreq
2
)
bottomfield
.
12. The apparatus of claim 3 , wherein the comb post-processing unit calculates a post-processed comb signal of a pixel based on a plurality of comb signals of neighboring pixels.
13. The apparatus of claim 12 , wherein the comb post-processing unit calculates the post-processed comb signal of the pixel according to the equations
c_mid1
=
(
∑
i
=
-
1
1
l
i
≥
DirCombCntThM
)
&
(
∑
i
=
-
1
1
(
h
i
>
0
)
≥
DirCombCntInRpwTh
)
c_mid2
=
(
min
(
C
i
-
1
,
j
′
C
i
,
j
′
C
i
+
1
,
j
′
)
≥
DirCombThH
)
c_low
=
(
∑
i
=
-
1
1
l
i
≥
DirCombCntThL
)
where
h
i
=
∑
j
=
-
4
4
(
C
i
,
j
′
>
DirCombThH
)
i
∈
[
-
1
,
1
]
l
i
=
∑
j
=
-
4
4
(
C
i
,
j
′
>
DirCombThL
)
i
∈
[
-
1
,
1
]
.
14. The apparatus of claim 2 , wherein the comb value estimation unit calculates a comb value of a pixel according to the equation
f
k
=
(
3
×
a
k
,
1
-
4
×
a
k
,
2
)
/
12
k
∈
[
0
,
2
]
where
a
k
,
1
=
∑
i
=
k
k
+
3
y
i
-
y
i
+
1
a
k
,
2
=
∑
i
=
k
k
+
2
y
i
-
y
i
+
2
.
15. The apparatus of claim 2 , wherein the comb value estimation unit calculates a comb value of a pixel according to the equations
f
k
=
(
3
×
a
k
,
1
-
4
×
a
k
,
2
)
/
12
k
∈
[
0
,
2
]
where
a
k
,
1
=
∑
i
=
k
k
+
3
y
i
-
y
i
+
1
a
k
,
2
=
∑
i
=
k
k
+
2
y
i
-
y
i
+
2
f
=
max
(
f
0
,
f
1
,
f
2
)
.
16. The apparatus of claim 4 , wherein the contrast adjustment unit is configured to calculate a contrast adjusted comb value for a low contrasted but highly confident comb pixel based on a comb value of a pixel.
17. The apparatus of claim 16 , wherein the contrast adjusted comb value is according to the equation
F
{
max
(
MinContras
t
,
f
)
max
(
b
0
,
b
1
,
b
2
)
≥
4
f
otherwise
where
b
k
=
∑
i
=
k
k
+
3
sign
(
y
i
,
y
i
+
1
,
DTh
)
k
∈
[
0
,
2
]
and
sign
(
y
i
,
y
i
+
1
,
DTh
)
=
{
1
y
odd
>
y
even
+
DTh
-
1
y
even
>
y
odd
+
DTh
0
otherwise
.
18. The apparatus of claim 1 , wherein the false motion excluded motion detector comprises:
a multiplexer configured to select a coupling field from a previous field and a next field to form a complete frame with a current field;
an inter-frame motion detection unit;
an inter-field motion detection unit coupled to the multiplexer; and
a false motion exclusion unit coupled to the inter-frame motion detection unit and the inter-field motion detection unit.
19. The apparatus of claim 18 , further comprising a motion post-processing unit coupled to the false motion exclusion unit.
20. The apparatus of claim 18 , wherein the current field, previous field, and next field are pre-processed by pre-filter units configured to provide phase-adjusted fields with noise immunity.
21. The apparatus of claim 18 , wherein the multiplexer is configured to select the coupling field according to a cadence and phase information provided by a film mode detector.
22. The apparatus of claim 18 , wherein the inter-frame motion detection unit is configured to receive the previous field and the next field and provide an inter-frame motion signal.
23. The apparatus of claim 18 , wherein the inter-field motion detection unit is configured to receive the coupling field and the current field and provide an inter-field motion signal.
24. The apparatus of claim 18 , wherein the false motion exclusion unit is configured to compare a received inter-field motion value and a pre-determined motion threshold and select the received inter-field motion value or ‘0’ according to a result of the comparison as a motion value.
25. The apparatus of claim 19 , wherein the motion post-processing unit is configured to receive a plurality of motion values and provide a post-processed motion detection signal according to the equation
m_mid
=
(
∑
i
=
-
1
1
m
i
≥
IFMotCntTh
)
&
(
∑
i
=
-
1
1
(
m
i
>
0
)
≥
IFMotCntInRowTh
)
where
m
i
=
∑
j
=
-
4
4
(
P
i
,
j
>
IFMotTh
)
i
∈
[
-
1
,
1
]
.
26. The apparatus of claim 1 , wherein the video fader value estimator comprises:
a pixel classification unit; and
a video value calculator coupled to the pixel classification unit.
27. The apparatus of claim 26 , further comprising a video value post-processing unit coupled to the video value calculator.
28. The apparatus of claim 26 , wherein the pixel classification unit is configured to receive a detected comb signal and a detected motion signal and provide a pixel classification result based on the received signals.
29. The apparatus of claim 26 , wherein the video value calculator is configured to receive a comb value, a motion value, and a pixel classification result and provide a video value based on the received comb value, motion value, and pixel classification result.
30. The apparatus of claim 27 , wherein the video value post-processing unit is configured to receive a plurality of video values and calculate a video fader value based on the plurality of video values.
31. The apparatus of claim 30 , wherein the video fader value is according to the equation
R
′
=
max
i
=
0
26
(
R
i
)
.
32. A local film/video mode processing apparatus, comprising:
a film mode detector;
a local video detector coupled to the film mode detector;
a film mode processing unit;
a video mode processing unit; and
a fader coupled to the local video detector, the film mode processing unit, and the video mode processing unit,
wherein the local video detector comprises:
a contour and contrast adjusted comb detector;
a false motion excluded motion detector; and
a video fader value estimator coupled to the contour and contrast adjusted comb detector and the false motion excluded motion detector.
33. The local film/video mode processing apparatus of claim 32 , wherein the film mode processing unit processes inverse telecine.
34. The local film/video mode processing apparatus of claim 32 , wherein the video mode processing unit processes de-interlacing.
35. A television (TV) and/or a set-top-box (STB) device comprising the local film/video mode processing apparatus of claim 32 .
36. A method of local film/video processing, comprising:
detecting a cadence and phase information from an input video;
calculating pixel-based video confidence values of the input video based on the cadence and phase information;
performing a film mode processing on the input video;
performing a video mode processing on the input video; and
fading between the film mode processing and the video mode processing according to the pixel-based video confidence values,
wherein calculating pixel-based video confidence values comprises:
calculating contour and contrast adjusted comb signals and a comb value for each pixel of the input video;
calculating false motion excluded motion signals and a motion value for each pixel of the input video; and
calculating pixel-based confidence values according to the comb signals, comb values, motion signals, and motion values.
37. The method of local film/video processing of claim 36 , wherein performing the film mode processing is performing inverse telecine.
38. The method of local film/video processing of claim 36 , wherein performing the video mode processing is performing de-interlacing.
39. The method of local film/video processing of claim 36 , wherein performing the film mode processing comprises:
receiving the input video;
interweaving coupling fields of the input video into merged video frames according to the cadence and phase information from an input video; and
outputting the merged video frames.
40. The method of local film/video mode processing of claim 36 , wherein performing the video mode processing comprises:
receiving the input video;
interpolating fields of the input video spatially and/or temporally to interpolated frames; and
outputting the interpolated frames.