IP Library Granted Patent US 11,295,423
Granted Patent B2
US 11,295,423 · App. 16/694,286 · Granted Apr 5, 2022

Unsupervised training of neural network for high dynamic range image compression

Inventors: Shlomo Bugdary (Rishon le Zion, IL); Rony Zatzarinni (Haifa, IL)
Assignee: Intel Corportation
G06T5/009G06N3/084G06T5/50G06T2207/20208
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,295,423
App. No.
16/694,286
Granted
Apr 5, 2022
Kind
B2
Abstract

Techniques are provided for unsupervised training of a neural network to perform compression of a high dynamic range (HDR) image. A methodology implementing the techniques according to an embodiment includes performing global tone mapping on an HDR training image to generate a low dynamic range (LDR) training image. The method also includes applying the neural network to the HDR training image and the LDR training image to generate a delta image representing image detail lost in the global tone mapping operation. The method further includes summing the delta image with the LDR training image to generate an output training image, and generating a loss function calculated from a weighted sum of a contrast loss and a compression loss. The contrast loss is based on the output training image and the HDR training image, and the compression loss is based on the output training image and the LDR training image.

Claims (41)

1. A processor-implemented method for training a neural network to perform high dynamic range (HDR) image compression, the method comprising:

performing, by a processor-based system, global tone mapping on an HDR training image to generate a low dynamic range (LDR) training image;

applying, by the processor-based system, the neural network to the HDR training image and the LDR training image, the neural network to generate a delta image, the delta image representing image detail lost between the HDR training image and LDR training image;

summing, by the processor-based system, the delta image with the LDR training image to generate an output training image;

calculating, by the processor-based system, a contrast loss, the contrast loss based on the output training image and the HDR training image;

calculating, by the processor-based system, a compression loss, the compression loss based on the output training image and the LDR training image; and

calculating, by the processor-based system, a loss function based on a weighted sum of the contrast loss and the compression loss.

2. The method of claim 1 , further including performing iterative backpropagation training of the neural network based on the loss function, such that the output training image converges to a compressed HDR image.

3. The method of claim 2 , wherein the weighted sum is based on a parameter selected to control a ratio of compression to preservation of contrast in the compressed HDR image.

4. The method of claim 1 , wherein the contrast loss is calculated as a rectilinear distance between a calculated contrast of the output training image and a calculated contrast of the HDR training image.

5. The method of claim 4 , wherein the calculated contrast of an image is calculated as a logarithm of a ratio of intensity of adjacent pixels of the image.

6. The method of claim 1 , wherein the compression loss is calculated as a rectilinear distance between a logarithm of the output training image and a logarithm of the LDR training image.

7. The method of claim 1 , wherein the neural network is a convolutional neural network.

8. The method of claim 1 , wherein the training of the neural network is unsupervised.

9. At least one non-transitory computer readable storage medium comprising instructions that, when executed, cause at least one processor to at least:

perform global tone mapping on an HDR training image to generate a low dynamic range (LDR) training image;

apply a neural network to the HDR training image and the LDR training image, the neural network to generate a delta image, the delta image representing image detail lost between the HDR training image and the LDR training image;

sum the delta image with the LDR training image to generate an output training image;

calculate a contrast loss, the contrast loss based on the output training image and the HDR training image;

calculate a compression loss, the compression loss based on the output training image and the LDR training image; and

calculate a loss function based on from a weighted sum of the contrast loss and the compression loss.

10. The computer readable storage medium of claim 9 , wherein the instructions, when executed, cause the at least one processor to perform iterative backpropagation training of the neural network based on the loss function, such that the output training image converges to a compressed HDR image.

11. The computer readable storage medium of claim 10 , wherein the weighted sum is based on a parameter selected to control a ratio of compression to preservation of contrast in the compressed HDR image.

12. The computer readable storage medium of claim 9 , wherein the contrast loss is calculated as a rectilinear distance between a calculated contrast of the output training image and a calculated contrast of the HDR training image.

13. The computer readable storage medium of claim 12 , wherein the calculated contrast of an image is calculated as a logarithm of a ratio of intensity of adjacent pixels of the image.

14. The computer readable storage medium of claim 9 , wherein the compression loss is calculated as a rectilinear distance between a logarithm of the output training image and a logarithm of the LDR training image.

15. The computer readable storage medium of claim 9 , wherein the neural network is a convolutional neural network.

16. The computer readable storage medium of claim 9 , wherein the training of the neural network is unsupervised.

17. An apparatus comprising:

memory;

instructions;

at least one processor to execute the instructions to:

perform global tone mapping on an HDR training image to generate a low dynamic range (LDR) training image;

apply the neural network to the HDR training image and the LDR training image, the neural network to generate a delta image, the delta image representing image detail lost between the HDR training image and the LDR training image;

sum the delta image with the LDR training image to generate an output training image;

calculate a contrast loss, the contrast loss based on the output training image and the HDR training image;

calculate a compression loss, the compression loss based on the output training image and the LDR training image; and

calculate a loss function based on a weighted sum of the contrast loss and the compression loss.

18. The apparatus of claim 17 , wherein the processor is to perform iterative backpropagation training of the neural network based on the loss function, such that the output training image converges to a compressed HDR image.

19. The apparatus of claim 18 , wherein the weighted sum is based on a parameter selected to control a ratio of compression to preservation of contrast in the compressed HDR image.

20. The apparatus of claim 17 , wherein the contrast loss is calculated as a rectilinear distance between a calculated contrast of the output training image and a calculated contrast of the HDR training image.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE FIRST INVENTOR'S NAME PREVIOUSLY RECORDED AT REEL: 051109 FRAME: 0910. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Oct 2, 2020
From: BUGDARY, SHLOMO; ZATZARINNI, RONY
To: INTEL CORPORATION
Reel/Frame 053969/0920 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 25, 2019
From: BUGDARY, SHLOMO; ZATZARINNI, RONY
To: INTEL CORPORATION
Reel/Frame 051109/0910 →
Continuity (1)
Related Publication 20200090313A1 · Mar 19, 2020