IP Library Granted Patent US 10,554,903
Granted Patent B2
US 10,554,903 · App. 16/368,974 · Granted Feb 4, 2020

Local tone mapping

Inventors: Guillaume Matthieu Guerin (Paris, FR); Antoine Regimbeau (Aigremont, FR); Thomas Nicolas Emmanuel Veit (Meudon, FR); Bruno César Douady-Pleven (Les Molieres, FR); Violaine Marie Mong-lan Sudret (Paris, FR)
Assignee: GoPro, Inc.
H04N5/2355G06T5/008H04N5/23229G06T5/009G06T5/50G06T2207/20028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,554,903
App. No.
16/368,974
Granted
Feb 4, 2020
Kind
B2
Abstract

Systems and methods are disclosed for image signal processing. For example, methods may include receiving an image from an image sensor; applying a filter to the image to obtain a low-frequency component image and a high-frequency component image; determining a first enhanced image based on a weighted sum of the low-frequency component image and the high-frequency component image, where the high-frequency component image is weighted more than the low-frequency component image; determining a second enhanced image based on the first enhanced image and a tone mapping; and storing, displaying, or transmitting an output image based on the second enhanced image.

Claims (65)

1. A system comprising:

an image sensor configured to capture an image; and

a processing apparatus configured to:

receive the image from the image sensor;

determine a perceptual domain image based on the image and a gamma curve that models human perception of contrast;

determine a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image;

determine an enhanced image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image; and

determine an output image based on the enhanced image, wherein the determination of the output image comprises the processing apparatus being configured to: determine gains for respective image portions based on the enhanced image and the gamma curve, and apply the gains for respective image portions to corresponding image portions of the image to obtain an output image.

2. The system of claim 1 , in which the determination of the low-frequency component perceptual domain image comprises the processing apparatus being configured to apply a transformation, based on the gamma curve, to a result of application of a tone mapping to a low-frequency component of the image.

3. The system of claim 1 , in which the determination of the low-frequency component perceptual domain image comprises the processing apparatus being configured to:

apply a bilateral filter to the image to obtain a low-frequency component image; and

apply a transformation, based on the gamma curve, to a result of applying a tone mapping to the low-frequency component of the image.

4. The system of claim 3 , in which the processing apparatus is configured to:

determine a reduced resolution image based on the image that is at a lower resolution than the image; and

in which applying the bilateral filter comprises processing pixels of the reduced resolution image as candidates.

5. The system of claim 3 , in which the application of the bilateral filter comprises the processing apparatus being configured to:

subsample candidates within a range of distances from a kernel center.

6. The system of claim 3 , in which the application of the bilateral filter comprises the processing apparatus being configured to:

subsample candidates at a first subsampling factor within a first range of distances from a kernel center; and

subsample candidates at a second subsampling factor within a second range of distances from the kernel center.

7. The system of claim 1 , in which the image sensor is attached to the processing apparatus.

8. A method comprising:

receiving an image;

determining a perceptual domain image based on the image and a gamma curve that models human perception of contrast;

determining a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image, in which determining the low-frequency component perceptual domain image comprises applying a transformation, based on the gamma curve, to a result of applying a tone mapping to a low-frequency component of the image;

determining an enhanced image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image; and

storing, displaying, or transmitting an output image based on the enhanced image.

9. The method of claim 8 , in which determining the output image comprises:

determining gains for respective image portions based on the enhanced image and the gamma curve; and

applying the gains for respective image portions to corresponding image portions of the image.

10. The method of claim 8 , in which determining the low-frequency component perceptual domain image comprises:

applying a bilateral filter to the image to obtain the low-frequency component image.

11. The method of claim 10 , comprising:

determine a reduced resolution image based on the image that is at a lower resolution than the image; and

in which applying the bilateral filter comprises processing pixels of the reduced resolution image as candidates.

12. The method of claim 10 , in which applying the bilateral filter comprises:

subsampling candidates within a range of distances from a kernel center.

13. The method of claim 10 , in which applying the bilateral filter comprises:

subsampling candidates at a first subsampling factor within a first range of distances from a kernel center; and

subsampling candidates at a second subsampling factor within a second range of distances from the kernel center.

14. A method comprising:

receiving an image from an image sensor;

applying a filter to the image to obtain a low-frequency component image and a high-frequency component image;

determining a non-linear mapping based on a histogram analysis of image portions of the of the low-frequency component image

applying the non-linear mapping to the low-frequency component image to obtain gains for respective image portions;

applying the gains for respective image portions to corresponding image portions of the image to obtain an enhanced image; and

storing, displaying, or transmitting an output image based on the enhanced image.

15. The method of claim 14 , in which the filter is a bilateral filter, and applying the bilateral filter comprises:

subsampling candidates at a first subsampling factor within a first range of distances from a kernel center; and

subsampling candidates at a second subsampling factor within a second range of distances from the kernel center.

16. The method of claim 14 , comprising:

determining a perceptual domain image based on the enhanced image and a gamma curve that models human perception of contrast;

determining a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image;

determining an enhanced perceptual domain image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image; and

wherein the output image is based on the enhanced perceptual domain image.

17. The method of claim 16 , in which determining the low-frequency component perceptual domain image comprises applying a transformation, based on the gamma curve, to a result of applying the gains for respective image portions to the low-frequency component image.

18. The method of claim 16 , in which determining the output image comprises:

determining gains for respective image portions based on the enhanced perceptual domain image and the gamma curve; and

applying the gains for respective image portions to corresponding image portions of the image.

19. The method of claim 14 , in which the filter is a bilateral filter, and applying the bilateral filter comprises:

determining a reduced resolution image based on the image that is at a lower resolution than the image; and

processing pixels of the reduced resolution image as candidates.

20. The method of claim 14 , in which determining the enhanced image comprises:

checking an underflow condition for an image portion of the enhanced image; and

where the underflow condition occurs, adding an offset to a corresponding image portion of the low-frequency component image, and subtracting the offset from a corresponding image portion of the high-frequency component image.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 14, 2019
From: GUERIN, GUILLAUME MATTHIEU; REGIMBEAU, ANTOINE; VEIT, THOMAS NICOLAS EMMANUEL; DOUADY-PLEVEN, BRUNO CESAR; SUDRET, VIOLAINE MARIE MONG-LAN
To: GOPRO, INC.
Reel/Frame 049175/0348 →
Continuity (2)
Continuation 15690772 · Aug 30, 2017
Related Publication 20190230274A1 · Jul 25, 2019
Cited By (2)
US 12,206,998 US 12,489,984