Detail reconstruction for SDR-HDR conversion
The exemplary embodiments relate to converting standard dynamic range (SDR) content to high dynamic range (HDR) content. An SDR image may be decomposed into a base layer of the SDR image that includes low frequency information from the SDR image and a detail layer of the SDR image that includes high frequency information from the SDR image. A base layer of an HDR image may be generated using the base layer of the SDR image and a detail layer of the HDR image may be generated using the detail layer of the SDR image. An HDR image is then generated using the base layer of the HDR image and the detail layer of the HDR image.
1. A method comprising:
receiving a standard dynamic range (SDR) image;
decomposing the SDR image into a base layer of the SDR image and a detail layer of the SDR image;
processing the base layer of the SDR image using a first neural network to generate a base layer of a high dynamic range (HDR) image;
generating a mask;
processing the mask and the detail layer of the SDR image using a second neural network to generate a coarse inpainting output used for generating a detail layer of the HDR image;
combining the base layer of the HDR image and the detail layer of the HDR image to generate a combined HDR output; and
processing the combined HDR output using a third fully neural network to generate the HDR image.
2. The method of claim 1 , wherein the first neural network is a fully convolutional autoencoder neural network, the second neural network is a partially convolutional autoencoder neural network, and the third neural network is another fully convolutional autoencoder neural network.
3. The method of claim 2 , wherein the first neural network is trained to convert an encoded representation of the base layer of the SDR image to the base layer of the HDR image by decoding an encoded representation of the base layer of the SDR image.
4. The method of claim 2 , wherein the third neural network is trained to convert an encoded representation of the combined HDR output to the HDR image by decoding the encoded representation of the combined HDR output.
5. The method of claim 1 , wherein processing the mask and the detail layer of the SDR image includes processing the coarse inpainting output using an inpainting network to generate the detail layer of the HDR image.
6. The method of claim 1 , wherein decomposing the SDR image includes:
filtering the SDR image to generate the base layer of the SDR image; and
dividing the SDR image by the base layer of the SDR image to generate the detail layer of the SDR image.
7. The method of claim 6 , wherein filtering the SDR image includes filtering the SDR image using a weighted least squares (WLS) filter.
8. A device comprising:
a communication interface; and
a processor configured to perform operations, the operations comprising:
receiving a standard dynamic range (SDR) image;
decomposing the SDR image into a base layer of the SDR image and a detail layer of the SDR image;
processing the base layer of the SDR image using a first neural network to generate a base layer of a high dynamic range (HDR) image;
generating a mask;
processing the mask and the detail layer of the SDR image using a second neural network to generate a coarse inpainting output used for generating a detail layer of the HDR image;
combining the base layer of the HDR image and the detail layer of the HDR image to generate a combined HDR output; and
processing the combined HDR output using a third fully neural network to generate the HDR image.
9. The device of claim 8 , wherein the first neural network is a fully convolutional autoencoder neural network, the second neural network is a partially convolutional autoencoder neural network, and the third neural network is another fully convolutional autoencoder neural network.
10. The device of claim 9 , wherein the first neural network is trained to convert an encoded representation of the base layer of the SDR image to the base layer of the HDR image by decoding an encoded representation of the base layer of the SDR image.
11. The device of claim 9 , wherein the third neural network is trained to convert an encoded representation of the combined HDR output to the HDR image by decoding the encoded representation of the combined HDR output.
12. The device of claim 8 , wherein processing the mask and the detail layer of the SDR image includes processing the coarse inpainting output using an inpainting network to generate the detail layer of the HDR image.
13. The device of claim 8 , wherein decomposing the SDR image includes:
filtering the SDR image to generate the base layer of the SDR image; and
dividing the SDR image by the base layer of the SDR image to generate the detail layer of the SDR image.
14. The device of claim 13 , wherein filtering the SDR image includes filtering the SDR image using a weighted least squares (WLS) filter.