Visual quality measure for real-time video processing
A measure of visual quality of processed images relative to unprocessed images is generated in real-time. The measure of visual quality closely correlates with a human's actual perception of the processed image relative to the original image. The measure of visual quality is computed based on a measure of discrepancy (e.g., mean square errors) between the processed and unprocessed images and the variance of each image in the pixel domain or the transform domain may be determined. If the processed image is unavailable, a prediction of the processed image may be used in place of the processed image. The prediction of a processed image may involve predicting the variance values for processed image blocks. The visual quality measure may be used in a feedback loop to improve processing or encoding.
1. A method comprising:
accessing a first set of transform domain variances for a first image and a second set of transform domain variances for a second image, the second image comprising a decompressed representation of the first image;
determining a mean squared error between the first image and the second image;
accessing a set of adaptation constants; and
computing a measure of visual quality of the second image based on the first and second sets of transform domain variances, the mean square error, the set of adaptation constants, and a number of bits that represent each pixel of the second image.
2. The method of claim 1 , wherein computing the measure of visual quality of the second image comprises:
computing a sub-measure of visual quality of the second image based at least in part on the first set of transform domain variances, the second set of transform domain variances, the mean square error, and the set of adaptation constants; and
computing the measure of visual quality of the second image based on the computed sub-measure of visual quality and the number of bits that represent each pixel of the second image.
3. The method of claim 2 , wherein the sub-measure of visual quality comprises a visual mean squared error.
4. The method of claim 1 , wherein the measure of visual quality comprises a visual peak signal-to-noise ratio.
5. The method of claim 1 , wherein the accessed set of adaptation constants is selected based on based on a comparison of a first set of images and a second set of images, the second set of images comprising decompressed representations of the first set of images.
6. The method of claim 1 , wherein computing the measure of visual quality of the second image comprises computing a sub-measure of visual quality for each of a plurality of blocks of the second image and averaging the computed sub-measures of visual quality to obtain the measure of visual quality.
7. The method of claim 1 , wherein the measure of visual quality is used by an image processor in the comparison of the first image and the second image.
8. A system comprising:
an image processor configured to:
access a first set of transform domain variances for a first image and a second set of transform domain variances for a second image, the second image comprising a decompressed representation of the first image;
determine a mean squared error between the first image and the second image;
access a set of adaptation constants; and
compute a measure of visual quality of the second image based on the first and second sets of transform domain variances, the mean square error, the set of adaptation constants, and a number of bits that represent each pixel of the second image.
9. The system of claim 8 , wherein computing the measure of visual quality of the second image comprises:
computing a sub-measure of visual quality of the second image based at least in part on the first set of transform domain variances, the second set of transform domain variances, the mean square error, and the set of adaptation constants; and
computing the measure of visual quality of the second image based on the computed sub-measure of visual quality and the number of bits that represent each pixel of the second image.
10. The system of claim 9 , wherein the sub-measure of visual quality comprises a visual mean squared error.
11. The system of claim 8 , wherein the measure of visual quality comprises a visual peak signal-to-noise ratio.
12. The system of claim 8 , wherein the accessed set of adaptation constants is selected based on based on a comparison of a first set of images and a second set of images, the second set of images comprising decompressed representations of the first set of images.
13. The system of claim 8 , wherein computing the measure of visual quality of the second image comprises computing a sub-measure of visual quality for each of a plurality of blocks of the second image and averaging the computed sub-measures of visual quality to obtain the measure of visual quality.
14. The system of claim 8 , wherein the measure of visual quality is used by an image processor in the comparison of the first image and the second image.
15. A non-transitory computer-readable storage medium storing executable computer program instructions for measuring a visual quality of a processed image, the instructions comprising instructions for:
accessing a first set of transform domain variances for a first image and a second set of transform domain variances for a second image, the second image comprising a decompressed representation of the first image;
determining a mean squared error between the first image and the second image;
accessing a set of adaptation constants; and
computing a measure of visual quality of the second image based on the first and second sets of transform domain variances, the mean square error, the set of adaptation constants, and a number of bits that represent each pixel of the second image.
16. The non-transitory computer-readable storage medium of claim 15 , wherein computing the measure of visual quality of the second image comprises:
computing a sub-measure of visual quality of the second image based at least in part on the first set of transform domain variances, the second set of transform domain variances, the mean square error, and the set of adaptation constants; and
computing the measure of visual quality of the second image based on the computed sub-measure of visual quality and the number of bits that represent each pixel of the second image.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the sub-measure of visual quality comprises a visual mean squared error.
18. The non-transitory computer-readable storage medium of claim 15 , wherein the measure of visual quality comprises a visual peak signal-to-noise ratio.
19. The non-transitory computer-readable storage medium of claim 15 , wherein the accessed set of adaptation constants is selected based on based on a comparison of a first set of images and a second set of images, the second set of images comprising decompressed representations of the first set of images.
20. The non-transitory computer-readable storage medium of claim 15 , wherein computing the measure of visual quality of the second image comprises computing a sub-measure of visual quality for each of a plurality of blocks of the second image and averaging the computed sub-measures of visual quality to obtain the measure of visual quality.
21. The non-transitory computer-readable storage medium of claim 15 , wherein the measure of visual quality is used by an image processor in the comparison of the first image and the second image.