System and a method for processing an image
A system and a method for processing an image. The system comprises an image gateway arranged to receive an input image showing a scene composed by a combination of a plurality of image portions of the input image, wherein one or more of the plurality of image portions is associated with an exposure level deviated from an optimal exposure level; and an enhancement engine arranged to process the input image by applying an exposure/image relationship to the input image, wherein the exposure/image relationship is arranged to adjust the exposure level of each of the plurality of image portions towards the optimal exposure level; and to generate an enhanced image showing a visual representation of the scene composed by a combination of the plurality of image portions of the input image with an adjusted exposure level.
1. A method for generating an HDR image from an input image I in comprising the steps of:
progressively inferring texture and structural features with learned confidence maps from two exposure gated detail recovering branches (EGDRBs), wherein one of the EGDRBs is adapted to process an image I o induced from the input image with an over-exposure level deviated from the optimal exposure level, and another EGDRB is adapted to process an image I u induced from the input image I in with under-exposure level deviated from the optimal exposure level;
processing the input image I in with a dynamic range expansion branch (DREB) that interacts with EGDRBs for expanding a global dynamic range of the input image I in with optimal exposure level; and
adaptively merging texture and structural features from the EGDRBs and features from DREB in a feature fusion module (FFM) to generate the HDR image.
2. A method for generating an HDR image of claim 1 , wherein the confidence maps are progressively generated in Exposure-Guided Confidence Map Learning Modules (ECMLMs) that are adapted to generate confidence maps in each scale for guiding the EGDRBs to progressively focus on extremely under/over-exposed regions of the input image.
3. A method for generating an HDR image of claim 2 , wherein the ECMLMs are embedded in the EGDRBs.
4. A method for generating an HDR image of claim 3 , wherein the EGDRBs are adapted to progressively infer missing texture and structural features in the under/over-exposed regions with the confidence maps learned from the ECMLMs.
5. A method for generating an HDR image of claim 4 , wherein the confidence maps are adapted to inferring appropriate details at a corresponding layer and delivering stable results for input images of different exposures.
6. A method for generating an HDR image of claim 5 , wherein the DREB takes I in as input to expand the dynamic range of the entire input image.
7. A method for generating an HDR image of claim 6 , wherein each of the confidence maps is trained by a learning network.
8. A method for generating an HDR image of claim 7 , wherein the machine learning network is a convolution neural network (CNN).
9. A method for generating an HDR image of claim 1 , wherein the DREB comprises a U-Net to expand the dynamic range of the input image.
10. A method for generating an HDR image of claim 9 , wherein the U-Net comprises a first convolutional layer and then four down-sample convolutional layers.
11. A method for generating an HDR image of claim 10 , wherein the DREB is adapted to receive the input image and subject to the first convolutional layer of the U-Net without down-sampling.
12. A method for generating an HDR image of claim 11 , wherein each of the four down-sample convolutional layers has a stride set to 2 to reduce spatial dimensions of each of the down-sample convolutional layers by half consecutively.
13. A method for generating an HDR image of claim 12 , wherein the convolutional layers comprise reflection padding for suppressing edge artifacts.
14. A method for generating an HDR image of claim 13 , wherein one or more convolutional layers comprise a spatial-channel attention module (SCAM) to adjust weights of features along spatial and channel dimensions.
15. A method for generating an HDR image of claim 14 , wherein the SCAM is adapted to generate two single-channel feature maps across channel dimensions using max pooling and average pooling operations from a feature map.
16. A method for generating an HDR image of claim 1 , wherein the FFM is arranged to combine features of image portions of the input image with the optimal exposure level and image portions associated with the over-exposure level or the under-exposure level with the texture and structural features inferred by the EGDRBs.
17. A method for generating an HDR image of claim 16 , wherein the features of the DREB and the texture and structural features inferred by EGDRBs are merged with skip connections.
18. A method for generating an HDR image of claim 17 , the DREB is adapted to expand the dynamic range of the input image by processing the image with one or more image quality loss processes.
19. A method for generating an HDR image of claim 18 , wherein the image quality loss processes comprise one or more of content loss, perceptual loss, or color loss processes.
20. A method for generating an HDR image of claim 19 , wherein the image quality loss processes are adapted to train a network for simultaneously expanding the dynamic range of the input image and reconstructing the details in the under- and over-exposed regions.