Frame packing and unpacking higher-resolution chroma sampling formats
View Patent ↗Video frames of a higher-resolution chroma sampling format such as YUV 4:4:4 are packed into video frames of a lower-resolution chroma sampling format such as YUV 4:2:0 for purposes of video encoding. For example, sample values for a frame in YUV 4:4:4 format are packed into two frames in YUV 4:2:0 format. After decoding, the video frames of the lower-resolution chroma sampling format can be unpacked to reconstruct the video frames of the higher-resolution chroma sampling format. In this way, available encoders and decoders operating at the lower-resolution chroma sampling format can be used, while still retaining higher resolution chroma information. In example implementations, frames in YUV 4:4:4 format are packed into frames in YUV 4:2:0 format such that geometric correspondence is maintained between Y, U and V components for the frames in YUV 4:2:0 format.
1. A computing device comprising:
one or more processing units;
volatile memory; and
non-volatile memory and/or storage, the non-volatile memory and/or storage having stored therein computer-executable instructions for causing the computing device, when programmed thereby, to perform operations comprising:
receiving a video frame of a higher-resolution format, the video frame of the higher-resolution format including sample values of first, second, and third component planes;
assigning the sample values of the first component plane of the video frame of the higher-resolution format to a first component plane of a first video frame of a lower-resolution format, the lower-resolution format having a lower resolution than the higher-resolution format;
assigning at least some of the sample values of the second and third component planes of the video frame of the higher-resolution format to second and third component planes of the first video frame of the lower-resolution format; and
assigning at least some of the sample values of the second and third component planes of the video frame of the higher-resolution format to first, second, and third component planes of a second video frame of the lower-resolution format.
2. The computing device of claim 1 , wherein the higher-resolution format is a 4:4:4 format and the lower-resolution format is a 4:2:0 format.
3. The computing device of claim 1 , wherein the sample values are in a color space selected from the group consisting of Y′UV, YIQ, Y′IQ, YdbDr, YCbCr, and YCoCg, the first component planes being luma, and the second and third component planes being chroma.
4. The computing device of claim 1 , wherein the sample values are in a color space selected from the group consisting of RGB and GBR.
5. The computing device of claim 1 , wherein the operations further comprise encoding the first and second video frames of the lower-resolution format.
6. The computing device of claim 5 , wherein the second assigning and the third assigning maintain geometric correspondence between adjacent sample values of the second and third component planes of the video frame of the higher-resolution format as adjacent sample values and/or collocated portions of the first and second video frames of the lower-resolution format, and wherein the encoding exploits the geometric correspondence.
7. The computing device of claim 6 , wherein the encoding exploits the geometric correspondence (a) in encoding operations that include one or more of derivation of motion vectors and derivation of prediction modes, and/or (b) to guide encoding decisions that include one or more of motion estimation, selection of quantization parameters and selection of prediction modes.
8. The computing device of claim 1 , wherein the first, second, and third assigning are consistent with plural design constraints, the plural design constraints including:
maintaining geometric correspondence among the sample values of the second and third component planes of the video frame of the higher-resolution format after the second assigning and the third assigning; and
embedding a lower-resolution version of the video frame of the higher-resolution format as the first video frame of the lower-resolution format.
9. The computing device of claim 1 , wherein the operations further comprise repeating the receiving, the first assigning, the second assigning, and the third assigning on a frame-by-frame basis for a sequence of video frames of the higher-resolution format.
10. A method of reconstructing a video frame of a higher-resolution format, the video frame of the higher-resolution format including sample values of first, second, and third component planes, the method comprising:
receiving first and second video frames of a lower-resolution format, the lower-resolution format having a lower resolution than the higher-resolution format;
assigning sample values of a first component plane of the first video frame of the lower-resolution format to the first component plane of the video frame of the higher-resolution format;
assigning at least some sample values of second and third component planes of the first video frame of the lower-resolution format to the second and third component planes of the video frame of the higher-resolution format; and
assigning at least some of the sample values of first, second, and third component planes of a second video frame of the lower-resolution format to the second and third component planes of the video frame of the higher-resolution format.
11. The method of claim 10 , wherein the higher-resolution format is a 4:4:4 format and the lower-resolution format is a 4:2:0 format.
12. The method of claim 10 , wherein the sample values are in a color space selected from the group consisting of Y′UV, YIQ, Y′IQ, YdbDr, YCbCr, and YCoCg, the first component planes being luma, and the second and third component planes being chroma.
13. The method of claim 10 , wherein the sample values are in a color space selected from the group consisting of RGB and GBR.
14. The method of claim 10 , further comprising decoding the first and second video frames of the lower-resolution format.
15. The method of claim 10 , further comprising repeating the receiving, the first assigning, the second assigning, and the third assigning on a frame-by-frame basis for a sequence of video frames of the higher-resolution format.