Display quality in a variable resolution video coder/decoder system
In a video coder/decoder system using variable resolution adaptation, decoder techniques provide a mechanism to changing resolution of coded lower-resolution video to a higher resolution for rendering. Coded video data of a low resolution frame may be decoded. A motion estimation search may be performed between the decoded low resolution frame and a cache of previously-stored high resolution video frames. If the motion estimation search generates one or more matches, high resolution video data of the decoded frame may generated as a derivation of matching data from the cached video frames.
1. A method of changing resolution of video, comprising:
decoding coded data of a current frame of the video to generate an initial decoded frame at a low resolution; and
altering resolution of one of the initial decoded frame and candidate high resolution frames to a resolution of the other of the initial decoded frame and the candidate high resolution frames;
after decoding operations are complete, for each candidate high resolution frame previously stored in a frame cache:
performing a motion estimation search between the initial decoded frame and the respective candidate high resolution frame
to identify a prediction reference block that matches content of a block of the initial decoded frame; and
generating a final decoded frame by blending together,
by a weighted averaging, contents of the initial decoded frame with matching content of the identified prediction reference block from the candidate frame;
wherein said final decoded frame has a resolution that is higher than the initial decoded frame.
2. The method of claim 1 , wherein the motion estimation is performed on each of a plurality of pixel blocks of the initial decoded frame.
3. The method of claim 2 , wherein the pixel blocks are spatially non-overlapping.
4. The method of claim 2 , wherein at least two pixel blocks spatially overlap.
5. The method of claim 1 , wherein the identified prediction reference block of a candidate high resolution frame is weighted according to error estimate(s) generated by the motion estimation search.
6. The method of claim 1 , wherein the identified prediction reference block of a candidate high resolution frame is weighted according to motion vector(s) generated by the motion estimation search.
7. The method of claim 1 , wherein the identified prediction reference block of a candidate high resolution frame is weighted according to a confidence score that based on an estimate[s] of a visual quality of the initial decoded frame or of the identified prediction reference block and that is stored with the identified prediction reference block of the candidate high resolution frame.
8. The method of claim 1 , wherein the blending further comprises blending DCT coefficients of the initial decoded frame and the identified prediction reference block of a candidate high resolution frame.
9. The method of claim 1 , further comprising storing the blended data contents in the frame cache.
10. The method of claim 1 , further comprising generating an estimate of a visual quality for the blended data contents based on a quality estimate obtained by the motion estimation, wherein the quality estimate is based on a level of residual coefficients that were generated following motion estimation.
11. The method of claim 1 , further comprising generating an estimate of a visual quality for the blended data contents based on a type of predictive coding that was applied to the initial decoded frame.
12. The method of claim 1 , further comprising generating an estimate of a visual quality for the blended data content based on a quantization assigned to the initial decoded frame.
13. The method of claim 1 , further comprising: wherein the altering comprises converting said the initial decoded frame to the high resolution before performing said motion estimation search.
14. The method of claim 1 , further comprising: wherein the altering comprises converting each candidate high resolution frame to the low resolution before performing said motion estimation search.
15. A non-transitory computer readable medium storing program instructions that, when executed by a processing device, cause the device to:
decode coded data of a current frame of video to generate an initial decoded frame at a low resolution;
alter resolution of one of the initial decoded frame and candidate high resolution frames to a resolution of the other of the initial decoded frame and the candidate high resolution frames;
after decoding operations are complete, for each of a plurality of stored high resolution frames:
perform a motion estimation search between the initial decoded frame and each of the respective candidate high resolution frame
to identify a prediction reference block that matches content of a block from the initial decoded frame; and
generate final frame data of the initial decoded frame by blending together,
by a weighted averaging contents of the of the initial decoded frame with the matching content of the identified prediction reference block from the candidate frame;
wherein said final frame data has a resolution that is higher than the initial decoded frame.
16. The medium of claim 15 , wherein the motion estimation is performed on each of a plurality of pixel blocks of the initial decoded frame.
17. The medium of claim 16 , wherein the pixel blocks are spatially non-overlapping.
18. The medium of claim 16 , wherein at least two pixel blocks spatially overlap.
19. The medium of claim 15 , wherein the identified block is weighted according to error estimate(s) generated by the motion estimation search.
20. The medium of claim 15 , wherein the identified block is weighted according to motion vector(s) generated by the motion estimation search.
21. The medium of claim 15 , wherein the identified block is weighted based on an estimate[s] of a visual quality of the initial decoded frame or of the identified prediction reference block and that is stored with the identified prediction reference block of a candidate high resolution frame.
22. The medium of claim 15 , wherein the blending further comprises blending DCT coefficients of the initial decoded frame and the identified prediction reference block of a stored high resolution frame.
23. The medium of claim 15 , further comprising storing the final frame data in a cache that stores the stored high resolution frames.
24. The medium of claim 15 , further comprising generating a confidence score for the final frame data based on a quality estimate obtained by the motion estimation.
25. An apparatus for changing resolution of video, comprising:
a memory device for storing frames of video data; and
a processor configured to change the resolution of a current frame of video data by:
decoding coded data of the current frame to generate an initial decoded frame at a low resolution;
altering resolution of one of the initial decoded frame and candidate high resolution frames to a resolution of the other of the initial decoded frame and the candidate high resolution frames;
after decoding operations are complete, for each candidate high resolution frame previously stored in the memory device:
performing a motion estimation search between the initial decoded frame and the respective candidate high resolution frame
to identify a prediction reference block that matches content of a block from the initial decoded frame; and
generating a final decoded frame by blending together,
by a weighted averaging contents of the initial decoded frame with the matching content of the identified prediction reference block from the candidate frame;
wherein said final decoded frame has a resolution that is higher than the initial decoded frame.
26. The apparatus of claim 25 , wherein the motion estimation is performed on each of a plurality of pixel blocks of the initial decoded frame.
27. The apparatus of claim 26 , wherein the pixel blocks are spatially non-overlapping.
28. The apparatus of claim 26 , wherein at least two pixel blocks spatially overlap.
29. The apparatus of claim 25 , wherein the initial decoded frame and the identified prediction reference blocks of the candidate high resolution frames are weighted according to error estimate(s) generated by the motion estimation search.
30. The apparatus of claim 25 , wherein the initial decoded frame and the identified prediction reference blocks of the candidate high resolution frames are weighted according to motion vector(s) generated by the motion estimation search.
31. The apparatus of claim 25 , wherein the initial decoded frame and the identified prediction reference blocks of the candidate high resolution frames are weighted according to confidence score stored with matching frame data.
32. The apparatus of claim 25 , wherein the blending comprises blending DCT coefficients of the initial decoded frame with an identified prediction reference block of a candidate high resolution frame.
33. The apparatus of claim 25 , further comprising storing the final frame data in the memory device.
34. The apparatus of claim 25 , further comprising generating a confidence score for the final frame data based on a quality estimate obtained by the motion estimation.