Method and system for objective quality assessment of image and video streams
View Patent ↗A method to predict visual quality of a DCT (discrete cosine transform) based compressed image or video stream without referring to its source. When applied to an MPEG video stream, the method is based on (1) an estimation of quantization errors using MPEG quantization scales and statistics of the inverse quantized DCT coefficients, (2) a blind estimation of the 8×8 and 16×16 blocking effect, and (3) an adaptive combination of the quantization error estimation and the blocking effect estimation using the MPEG motion vector information. The method may be used in many applications, such as network video servers, switches and multiplexers for automatic quality monitoring and control of video services, video encoders, decoders, transcoders, and statistical multiplexers for picture quality optimization.
1. A method for measuring the video quality of a coded frame of a compressed video stream without access to an original or reference video source and while utilizing only the information embedded in the compressed video stream, comprising:
estimating quantization errors of the coded frame of the compressed video stream;
estimating blocking effects of the coded frame of the compressed video stream; and
measuring the video quality of the coded frame of the compressed video stream by adaptively combining the estimate of quantization errors with the estimate of blocking effects and with motion information on the coded frame of the compressed video stream; wherein: the compressed video stream comprises an MPEG video stream consisting of a sequence of coded frames;
an input MPEG video bitstream is partially decoded by an MPEG decoder to obtain (1) inverse quantized DCT coefficients, (2) quantization scale and (3) motion vectors for each frame; and
the step of estimating quantization errors includes the step of estimating a probability distribution of the inverse quantized DCT coefficients by applying histogram statistics to the inverse quantized DCT coefficients to derive a histogram, which is then used to estimate a distribution on a piece-wise basis.
2. A method for measuring the video quality of a coded frame of a compressed video stream without access to an original or reference video source and while utilizing only the information embedded in the compressed video stream, comprising:
estimating quantization errors of the coded frame of the video stream;
estimating blocking effects of the coded frame of the video stream;
measuring the video quality of the coded frame of the video stream by adaptively combining the estimate of quantization errors with the estimate of blocking effects and with motion information on the coded frame of the compressed video stream, wherein the compressed video stream comprises an MPEG video stream consisting of a sequence of coded frames;
said MPEG video stream is partially decoded by an MPEG decoder to obtain (1) inverse quantized DCT coefficients, (2) quantization scale and (3) motion vectors for each frame;
and
a probability distribution of inverse quantized DCT coefficients is estimated by applying histogram statistics to inverse quantized DCT coefficients, which are obtained from the MPEG decoder, to derive a histogram which is then used to estimate a distribution on a piece-wise basis, wherein for a given DOT coefficient, if the inverse quantized value L and the quantization scale is q, then the quantization error is estimated as
E
=
∫
L
-
q
/
2
L
+
q
/
2
x
-
L
2
p
(
x
)
ⅆ
x
∫
L
-
q
/
2
L
+
q
/
2
p
(
x
)
ⅆ
x
where p(x) is the estimated probability distribution of the DCT coefficient, and the quantization errors of all the inverse quantized coefficients are then averaged together to obtain an estimate of the overall quantization error of the frame.
3. The method for measuring the video quality of a coded frame of a compressed video stream without access to an original or reference video source and while utilizing only the information embedded in the compressed video stream, comprising:
estimating quantization errors of the coded frame of the compressed video stream;
estimating blocking effects of the coded frame of the compressed video stream; and
measuring the video quality of the coded frame of the compressed video stream by adaptively combining the estimate of quantization errors with the estimate of blocking effects and with motion information on the coded frame of the compressed video stream;
wherein in estimating blocking effects, for a row f(n), n=1, 2, . . . from the image, take the absolute difference of adjacent samples, d(n)=|f(n)−f(n−1)|, wherein a blocky signal d(n) has periodic peaks, detect the periodic peaks and use the relative magnitude between the peaks in d(n) and their adjacent samples of d(n) as an estimate of the blocking effects, performed on all or a proportion of all the rows and columns, and average the blockiness values to obtain an overall blocking effect measure of the frame.
4. The method for measuring the quality of a coded frame of a compressed image stream without access to an orignal or reference image source and while utilizing only the information embedded in the compressed image stream, comprising:
estimating quantization errors of the coded frame of the compressed image stream; estimating blocking effects of the coded frame of the compressed image stream; and
measuring the video quality of the coded frame of the compressed image stream by combining the estimate of quantization errors with the estimate of blocking effects;
wherein:
the compressed image stream comprises a JPEG coded image stream; and
the step of estimating the quantization errors uses JPEG quantization scales and statistics of inverse quantized DCT coefficients, including the step of estimating a probability distribution of the inverse quantized DCT coefficients by applying histogram statistics to the inverse quantized DCT coefficients to derive a histogram, which is then used to estimate a distribution on a price-wise basis.
5. A method for measuring the quality of a coded frame of a compressed image stream without access to an original or reference image source and while utilizing only the information embedded in the compressed image stream, comprising:
estimating quantization errors of the coded frame of the image stream;
estimating blocking effects of the coded frame of the image stream; and
measuring the video quality of the coded frame of the image stream by combining the estimate of quantization errors with the estimate of blocking effects; wherein
the compressed image stream comprises a JPEG coded image stream;
the step of estimating the quantization errors uses JPEG quantization scales and statistics of inverse quantized DCT coefficients; and
a probability distribution of inverse quantized DCT coefficients is estimated by applying histogram statistics to inverse quantized DCT coefficients, to derive a histogram which is then used to estimate a distribution on a piece-wise basis, wherein for a given DCT coefficient, if the inverse quantized value L and the quantization scale is q, then the quantization error is estimated as
E
=
∫
L
-
q
/
2
L
+
q
/
2
x
-
L
2
p
(
x
)
ⅆ
x
∫
L
-
q
/
2
L
+
q
/
2
p
(
x
)
ⅆ
x
where p(x) is the estimated probability distribution of the DCT coefficient, and the quantization errors of all the inverse quantized coefficients are then averaged together to obtain an estimate of the overall quantization error of the frame.
6. The method of claim 5 , wherein in estimating blocking effects, for a row f(n), n=1,2, . . . from the image, take the absolute difference of adjacent samples, d(n)=|f(n)−f(n−1)|, wherein a blocky signal d(n) has periodic peaks, detect the periodic peaks and use the relative magnitude between the peaks in d(n) and their adjacent samples of d(n) as an estimate of the blocking effects, performed on all or a proportion of all the rows and columns, and average the blockiness values to obtain an overall blocking effect measure of the frame.