System for and method of sharpness enhancement for coded digital video
A method and system for calculating a UME for each pixel in a frame of video data. A system for enhancing sharpness of a coded digital video signal representative a frame of block-based data is provided, comprising: a system for calculating a UME of each pixel in the frame in accordance with a frame type, wherein the calculating system includes: a mechanism that defines local spatial features in the frame; a mechanism that analyzes block-based motion estimation for the frame; a mechanism that detects a scene change in the frame; and a sharpness enhancer that improves sharpness to a decoded version of the digital video signal based on said UME.
1. A system for enhancing sharpness of a coded digital video signal representative of a frame of block-based data, said system comprising:
a system for calculating a usefulness metric for enhancement (UME) of each pixel in the frame in accordance with a frame type, wherein the calculating system includes:
a mechanism that defines local spatial features in the frame;
a mechanism that analyzes block-based motion estimation for the frame; and
a mechanism that detects a scene change in the frame; and
a sharpness enhancer that improves sharpness to a decoded version of the digital video signal based on said UME.
2. A system for enhancing sharpness of a coded digital video signal representative of a frame of block-based data, said system comprising:
a system for calculating a usefulness metric for enhancement (UME) of each pixel in the frame in accordance with a frame type, wherein the calculating system includes:
a mechanism that defines local spatial features in the frame;
a mechanism that analyzes block-based motion estimation for the frame; and
a mechanism that detects a scene change in the frame; and
a sharpness enhancer that improves sharpness to a decoded version of the digital video signal based on said UME,
wherein the calculating system further includes a mechanism that analyzes macroblock and block based coding information according the formula:
var
(
i
,
j
)
=
∑
k
=
-
q
q
∑
m
=
-
q
q
pix
(
i
+
k
,
j
+
m
)
-
mean
wherein UME_block is a usefulness metric for a block of pixel data, q_scale is a quantization scale for a macroblock, num_bits is a number of bits to encode a luminance block, max is a function representing a maximum value for the frame, and M and N are scaling factors.
3. The system of claim 2 , wherein the local spatial features are defined as a variance of pixel luminance values over an n×n window covering n×n pixels, said variance defined according to the equation:
var
(
i
,
j
)
=
∑
k
=
-
q
q
∑
m
=
-
q
q
pix
(
i
+
k
,
j
+
m
)
-
mean
wherein q=(n−1)/2, pix(i+k, j+m) is a pixel value at location (i+k, j+m) and mean is an average pixel value over said n×n window.
4. The system of claim 3 , wherein the UME of a pixel in an I frame is calculated using UME_block if:
(var(i,j)≧VAR_THRED) and (num_bits≠0); and
a scene change is detected,
wherein VAR_THRED is a predetermined threshold value.
5. The system of claim 3 , wherein the mechanism that analyzes block-based motion estimation for the frame determines if the block is a still block.
6. The system of claim 3 , wherein the mechanism that analyzes block-based motion estimation for the frame calculates a motion vector (v′,h′) if a current block is not a still block, wherein the motion vector (v′,h′) is a motion vector in a block B({right arrow over (X)}) having a smallest summed absolute difference (SAD) in a search window, and wherein the summed absolute difference for a vector {right arrow over (V)} in a frame n is defined as:
SAD
(
V
→
,
X
→
,
n
)
=
∑
x
→
∈
B
(
X
→
)
f
(
x
→
,
n
)
-
f
(
x
→
-
V
→
,
n
-
1
)
where ƒ({right arrow over (x)},n) is a pixel value at location {right arrow over (x)}.
7. The system of claim 6 , wherein if the block is not a still block, the calculating system tests to determine if both v′ and h′ equal zero.
8. The system of claim 7 , wherein if the block is not a still block, the calculating system sets the UME to zero if var<VAR_THRED, wherein VAR_THRED is a predetermined threshold value.
9. The system of claim 8 , wherein if the block is not a still block and both v′ and h′ equal zero, the calculating system sets the UME to UME=[0.1*UME]+[0.9*UME_prev] if var≧VAR_THRED, where UME_prev is the UME value of a pixel at the same location in a previous frame.
10. The system of claim 8 , wherein if the block is not a still block and if at least one of v′ and h′ do not equal zero, the calculating system sets the UME to UME=[0.9*UME]+[0.1*UME_prev(v′, h′)], if var≧VAR_THRED, where UME_prev(v′, h′) is the UME value of the pixel pointed by the vector (v′, h′) in the previous frame.
11. The system of claim 1 , wherein said sharpness enhancement algorithm is a peaking algorithm.
12. The system of claim 1 , wherein said sharpness enhancement algorithm is a spatial-domain algorithm.
13. A system for enhancing sharpness of a coded digital video signal representative of a frame of block-based data, said system comprising:
a system for calculating a usefulness metric for enhancement (UME) of each pixel in the frame in accordance with a frame type, wherein the calculating system includes:
a mechanism that defines local spatial features in the frame;
a mechanism that analyzes block-based motion estimation for the frame; and
a mechanism that detects a scene change in the frame; and
a sharpness enhancer that improves sharpness to a decoded version of the digital video signal based on said UME,
wherein the sharpness enhancement algorithm applied is:
out_pixel=input_pixel+ UME *convolution_result,
wherein output_pixel is a final luminance value to be used for the pixel, input_pixel is a luminance value of the input pixel, and convolution_result is a high-pass filter output.
14. A method for enhancing sharpness of a coded digital video signal representative of a frame of block-based data, the method comprising:
defining local spatial features in the frame;
analyzing block-based motion estimation for the frame;
detecting a scene change in the frame;
calculating a usefulness metric for enhancement (UME) of each pixel in the frame based on local spatial features block-based motion estimation and detected scene changes; and
improving sharpness of the frame by applying the UME calculated for each pixel.
15. A method for enhancing sharpness of a coded digital video signal representative of a frame of block-based data, the method comprising:
defining local spatial features in the frame;
analyzing block-based motion estimation for the frame;
detecting a scene change in the frame;
calculating a usefulness metric for enhancement (UME) of each pixel in the frame based on local spatial features, block-based motion estimation and detected scene changes; and
improving sharpness of the frame by applying the UME calculated for each pixel,
wherein the calculating step is further based on a mechanism that analyzes macroblock coding information according the formula:
UME_block
=
1
-
M
*
(
1
+
q_scale
N
)
2
*
q_scale
num_bits
max
(
q_scale
num_bits
)
wherein UME_block is a usefulness metric for a block of pixel data, q_scale is a quantization scale for a macroblock, num_bits is a number of bits to encode a luminance block, max is a function representing a maximum value for the frame, and M and N are scaling factors.
16. The method of claim 15 , wherein the local spatial features for a pixel at location (i,j) is defined as a variance of pixel luminance values over an n×n window covering n×n pixels, said variance defined according to the equation:
var
(
i
,
j
)
=
∑
k
=
-
q
q
∑
m
=
-
q
q
pix
(
i
+
k
,
j
+
m
)
-
mean
wherein q=(n−1)/2, pix(i+k, j+m) is a pixel value at location (i+k, j+m) and mean is an average pixel value over said n×n window.
17. The method of claim 16 , wherein the UME of a pixel in an I frame is calculated using UME_block if:
(var(i,j)≧VAR_THRED) and (num_bits≠0); and
a scene change is detected,
wherein VAR_THRED is a predetermined threshold value.
18. The method of claim 16 , wherein the UME of a pixel in a current block of a P and B frame is calculated using UME_block if:
a scene change is detected;
num_bits do not equal zero; and
the current block is an intrablock.
19. The method of claim 16 , wherein the calculating step includes the steps of:
determining if a scene change occurs in the frame; and
if no scene change occurs, using the mechanism that analyzes block-based motion estimation for the frame to determine if a current block is a still block.
20. The method of claim 19 , wherein the calculating step includes the further step of:
if the current block is not a still block, using the mechanism that analyzes block-based motion estimation for the frame to calculate a motion vector (v′,h′), wherein the motion vector (v′,h′) is a motion vector in a block B({right arrow over (X)}) having a smallest summed absolute difference (SAD) in a search window, and wherein the summed absolute difference for a vector {right arrow over (V)} in a frame n is defined as:
SAD
(
V
→
,
X
→
,
n
)
=
∑
x
→
∈
B
(
X
→
)
f
(
x
→
,
n
)
-
f
(
x
→
-
V
→
,
n
-
1
)
where ƒ({right arrow over (x)},n) is a pixel value at location {right arrow over (x)}.
21. The method of claim 20 , wherein the calculating step includes the further step of:
determining if both v′ and h′ equal zero.
22. The method of claim 21 , wherein the calculating step includes the further step of: testing if var≧VAR_THRED, wherein VAR_THRED is a predetermined threshold value.
23. The method of claim 22 , wherein, if v′ and h′ both equal zero, the calculating step includes the further steps of:
setting the UME to zero if var<VAR_THRED; and
setting the UME to UME_prev if var≧VAR_THRED, where UME_prev is the UME value of a pixel at the same location in the previous frame.
24. The method of claim 22 , wherein, if at least one of v′ and h′ do not equal zero, the calculating step includes the further steps of:
setting the UME to zero if var<VAR_THRED; and
setting the UME to UME_prev(v′,h′) if var≧VAR_THRED, where UME_prev(v′,h′) is the UME of a pixel pointed by the motion vector (v′,h′) in the previous frame.
25. The method of claim 14 , wherein the step of improving sharpness is implemented using the formula:
out_pixel=input_pixel+ UME *convolution_result,
wherein output_pixel is a final luminance value to be used for the pixel, input_pixel is a luminance value of the input pixel, and convolution_result is a high-pass filter output.
26. A program product stored on a recordable medium for calculating a usefulness metric for enhancement (UME) for each pixel in a frame of video data, the program product comprising:
means for defining local spatial features in the frame;
means for analyzing block-based motion estimation for the frame;
means for detecting a scene change in the frame; and
means for calculating the UME using the defining means, analyzing means, and detecting means.
27. A program product stored on a recordable medium for calculating a usefulness metric for enhancement (UME) for each pixel in a frame of video data, the program product comprising:
means for defining local spatial features in the frame;
means for analyzing block-based motion estimation for the frame;
means for detecting a scene change in the frame; and
means for calculating the UME using the defining means, analyzing means, and detecting means,
wherein the means for defining local spatial features in the frame defines a variance of pixel luminance values over an n×n window covering n×n pixels for a pixel at location (i, j), said variance defined according to the equation:
var
(
i
,
j
)
=
∑
k
=
-
q
q
∑
m
=
-
q
q
pix
(
i
+
k
,
j
+
m
)
-
mean
wherein q=(n−1)/2, pix(i+k, j+m) is a pixel value at location (i+k, j+m) and mean is an average pixel value over said n×n window.
28. The program product of claim 26 , wherein the means for analyzing block-based motion estimation for the frame includes means for determining if a current block is a still block.
29. A program product stored on a recordable medium for calculating a usefulness metric for enhancement (UME) for each pixel in a frame of video data, the program product comprising:
means for defining local spatial features in the frame;
means for analyzing block-based motion estimation for the frame,
means for detecting a scene change in the frame; and
means for calculating the UME using the defining means, analyzing means, and detecting means,
wherein the means for analyzing block-based motion estimation for the frame further includes:
means for calculating a motion vector (v′,h′), wherein the motion vector (v′,h′) is a motion vector in a block B({right arrow over (X)}) having a smallest summed absolute difference (SAD) in a search window, and wherein the summed absolute difference for a vector {right arrow over (V)} in a frame n is defined as:
SAD
(
V
→
,
X
→
,
n
)
=
∑
x
→
∈
B
(
X
→
)
f
(
x
→
,
n
)
-
f
(
x
→
-
V
→
,
n
-
1
)
where ƒ({right arrow over (x)},n) is a pixel value at location {right arrow over (x)}.