Backward-compatible HDR codecs with temporal scalability
A processor for video coding receives a full-frame rate (FFR) HDR video signal and a corresponding FFR SDR video signal. An encoder generates a scalable bitstream that allows decoders to generate half-frame-rate (HFR) SDR, FFR SDR, HFR HDR, or FFR HDR signals. Given odd and even frames of the input FFR SDR signal, the scalable bitstream combines a base layer of coded even SDR frames with an enhancement layer of coded packed frames, where each packed frame includes a downscaled odd SDR frame, a downscaled even HDR residual frame, and a downscaled odd HDR residual frame. In an alternative implementation, the scalable bitstream combines four signals layers: a base layer of even SDR frames, an enhancement layer of odd SDR frames, a base layer of even HDR residual frames and an enhancement layer of odd HDR residual frames. Corresponding decoder architectures are also presented.
1. A method for backward compatible encoding of HDR images, the method comprising:
receiving first and second high-dynamic range (HDR) images;
receiving first and second standard dynamic range (SDR) images corresponding to the first and second HDR images, wherein the first and second HDR images are two consecutive frames in a full-frame rate HDR video signal, and the first and second SDR images are two consecutive frames in a corresponding full-frame rate HDR video signal;
encoding the first SDR image with an encoder to generate a first coded image;
down-sampling the second SDR image to generate a down-sampled SDR image;
generating a first predicted HDR image based on the first coded image;
generating a second predicted HDR image based on the down-sampled SDR image;
generating a first residual image based on the first predicted HDR image and the first HDR image;
generating a second residual image based on the second predicted HDR image and the second HDR image;
generating a packed image based on the first residual image, the second residual image, and the down-sampled SDR image;
encoding the packed image with an encoder to generate a second coded image; and
multiplexing the first coded image and the second coded image to generate an output coded image.
2. The method of claim 1 , wherein generating the packed image further comprises:
down-sampling the second SDR image by a factor of two in either the horizontal or vertical dimension;
down-sampling the first residual image by a factor of two in both the horizontal and the vertical dimensions to generate a first down-sampled residual image;
down-sampling the second residual image by a factor of two in both the horizontal and the vertical dimensions to generate a second down-sampled residual image; and
generating the packed image based on the down-sampled second SDR image, the first down-sampled residual image, and the second down-sampled residual image.
3. The method of claim 2 , wherein if the second SDR image is down-sampled in the horizontal direction then the first and the second down-sampled residual images are packed in a top-bottom configuration.
4. The method of claim 2 , wherein if the second SDR image is down-sampled in the vertical direction then the first and the second down-sampled residual images are packed in a side-by-side configuration.
5. The method of claim 2 , wherein the encoding of the packed image comprises using tiles, wherein each of the down-sampled images in the packed image is encoded independently of the others.
6. The method of claim 1 , wherein the output coded image further comprises metadata related to parameters used to generate the predicted HDR images.
7. A method of decoding with a decoder, the method comprising:
receiving a coded bitstream comprising a coded temporally-scalable SDR stream, a coded temporally-scalable HDR residual stream, and a metadata stream;
decoding the coded temporally-scalable SDR stream to generate a first SDR video signal in a half frame rate and a second SDR video signal in a half frame rate; and
combining the first SDR video signal and the second SDR video signal to generate an output SDR video signal at full frame rate,
decoding the coded temporally-scalable HDR residual stream to generate a first HDR residual signal and a second HDR residual signal;
generating a first predicted HDR signal based on the first SDR video signal;
generating a second predicted HDR signal based on the second SDR video signal;
generating a first half-frame rate HDR signal based on the first predicted HDR signal and the first HDR residual signal; generating a second half-frame rate HDR signal based on the second predicted HDR signal and the second HDR residual signal; and
combining the first HDR video signal and the second HDR video signal to generate an output HDR video signal at a full frame rate.
8. An apparatus comprising a processor and configured to perform the method recited in claim 1 .
9. A non-transitory computer-readable storage medium having stored thereon computer-executable instruction for executing a method with one or more processors in accordance with claim 1 .