IP Library › Granted Patent US 11,689,814
Granted Patent B1
US 11,689,814 · App. 17/540,468 · Granted Jun 27, 2023

System and a method for processing an image

Inventors: Sam Tak Wu Kwong (Hong Kong, CN); Zhangkai Ni (Hong Kong, CN); Yue Liu (Hong Kong, CN); Shiqi Wang (Hong Kong, CN)
Assignee: CENTRE FOR INTELLIGENT MULTIDIMENSAIONAL DATA ANALYSIS LIMITED
H04N23/741G06N3/08G06T5/009G06T5/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,689,814
App. No.
17/540,468
Granted
Jun 27, 2023
Kind
B1
Abstract

A system and a method for processing an image. The system comprises an image gateway arranged to receive an input image showing a scene composed by a combination of a plurality of image portions of the input image, wherein one or more of the plurality of image portions is associated with an exposure level deviated from an optimal exposure level; and an enhancement engine arranged to process the input image by applying an exposure/image relationship to the input image, wherein the exposure/image relationship is arranged to adjust the exposure level of each of the plurality of image portions towards the optimal exposure level; and to generate an enhanced image showing a visual representation of the scene composed by a combination of the plurality of image portions of the input image with an adjusted exposure level.

Claims (25)

1. A method for processing an image comprising the steps of:

receiving an input image showing a scene composed by a combination of a plurality of image portions of the input image, wherein one or more of the plurality of image portions is associated with an exposure level deviated from an optimal exposure level;

processing the input image by applying an exposure/image relationship to the input image, wherein the exposure/image relationship is arranged to adjust the exposure level of each of the plurality of image portions towards the optimal exposure level; and

generating an enhanced image showing a visual representation of the scene composed by a combination of the plurality of image portions of the input image with an adjusted exposure level,

wherein one or more of the plurality of image portions is further associated with loss of details in a visual representation of the image due to an over-exposure level or an under-exposure level associated with the corresponding image portions of the image,

wherein the step of processing the input image by applying the exposure/image relationship to the input image comprises the step of recovering visual details in the image portions associated with the over-exposure level or the under-exposure level, and

wherein the step of processing the input image by applying the exposure/image relationship to the input image comprises the step of processing gated images lo or lu indicating respectively the image portions associated with the over-exposure level or the under-exposure level with a confidence map Mo or Mu, to determine a probability of information loss in the corresponding image portion.

2. The method for processing an image in accordance with claim 1 , wherein the confidence map is further represented by Moi or Mui of multiple scales i indicating the level of over-exposure or under-exposure.

3. The method for processing an image in accordance with claim 2 , wherein the confidence map Moi or Mui¬ is represented by Wf×Wm, wherein Wf denotes a feature weight map obtained by passing an output feature map associated with the (i−1)th scale through a convolution layer and a Sigmoid function, and Wm denotes a down-sampled confidence map of the ith scale obtained by average pooling operation of a confidence map of the (i−1)th scale.

4. The method for processing an image in accordance with claim 3 , wherein the confidence map is trained by a learning network.

5. The method for processing an image in accordance with claim 4 , wherein the learning network is a convolution neural network (CNN).

6. The method for processing an image in accordance with claim 5 , wherein the learning network has a progressive learning structure.

7. The method for processing an image in accordance with claim 6 , wherein the step of processing the input image by applying the exposure/image relationship to the input image comprises the step of progressively recovering visual details of different image portions associated with different over-exposure levels or different under-exposure level with confidence maps Moi or Mui of multiple scales i.

8. The method for processing an image in accordance with claim 7 , wherein the step of processing the input image by applying the exposure/image relationship to the input image comprises the step of expanding a dynamic range of the input image.

9. The method for processing an image in accordance with claim 8 , wherein the step of expanding the dynamic range of the input image comprises the step of concatenating features of remaining image portions of the input image with the optimal exposure level and the image portions associated with the over-exposure level or the under-exposure level with recovered visual details.

10. The method for processing an image in accordance with claim 9 , wherein the step of generating the enhanced image further comprising the step of generating a high dynamic range (HDR) image based on the input image of a standard dynamic range (SDR) image.

11. The method for processing an image in accordance with claim 10 , wherein the step of expanding the dynamic range of the input image comprises the step of processing the image with one or more image quality loss processes.

12. The method for processing an image in accordance with claim 11 , wherein the one or more image quality loss processes include content loss, perceptual loss, color loss or any combination thereof.

13. A system for processing an image comprising:

an image gateway arranged to receive an input image showing a scene composed by a combination of a plurality of image portions of the input image, wherein one or more of the plurality of image portions is associated with an exposure level deviated from an optimal exposure level; and

an enhancement engine arranged to process the input image by applying an exposure/image relationship to the input image, wherein the exposure/image relationship is arranged to adjust the exposure level of each of the plurality of image portions towards the optimal exposure level; and to generate an enhanced image showing a visual representation of the scene composed by a combination of the plurality of image portions of the input image with an adjusted exposure level,

wherein one or more of the plurality of image portions is further associated with loss of details in a visual representation of the image due to an over-exposure level or an under-exposure level associated with the corresponding image portions of the image,

wherein the enhancement engine comprises an exposure gated detail recovering module arranged to recover visual details in the image portions associated with the over-exposure level or the under-exposure level,

wherein the enhancement engine further comprises a dynamic range expansion module arranged to expand a dynamic range of the input image,

wherein the dynamic range expansion module includes a feature fusion module arranged to combine features of remaining image portions of the input image with the optimal exposure level and the image portions associated with the over-exposure level or the under-exposure level with visual details recovered by the exposure gated detail recovering module.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2022
From: KWONG, SAM TAK WU; NI, ZHANGKAI; LIU, YUE; WANG, SHIQI
To: CENTRE FOR INTELLIGENT MULTIDIMENSIONAL DATA ANALYSIS LIMITED
Reel/Frame 059180/0649 →