IP Library Granted Patent US 11,178,341
Granted Patent B2
US 11,178,341 · App. 16/780,132 · Granted Nov 16, 2021

Local tone mapping

Inventors: Guillaume Matthieu Guerin (Chatillon, FR); Antoine Regimbeau (Aigremont, FR); Thomas Nicolas Emmanuel Veit (Meudon, FR); Bruno Cesar Douady-Pleven (Orsay, FR); Violaine Marie Mong-lan Sudret (Chatillon, FR)
Assignee: GoPro, Inc.
H04N5/2355G06T5/008H04N5/23229G06T5/009G06T5/50G06T2207/20028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,178,341
App. No.
16/780,132
Granted
Nov 16, 2021
Kind
B2
Abstract

Systems and methods are disclosed for image signal processing. For example, methods may include receiving an image from an image sensor; applying a filter to the image to obtain a low-frequency component image and a high-frequency component image; determining a first enhanced image based on a weighted sum of the low-frequency component image and the high-frequency component image, where the high-frequency component image is weighted more than the low-frequency component image; determining a second enhanced image based on the first enhanced image and a tone mapping; and storing, displaying, or transmitting an output image based on the second enhanced image.

Claims (64)

1. A system comprising:

an image sensor configured to capture an image; and

a processing apparatus configured to:

receive the image from the image sensor;

determine a perceptual domain image based on the image and a gamma curve that models human perception of contrast;

determine a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image, in which the determination of the low-frequency component perceptual domain image comprises the processing apparatus being configured to apply a transformation, based on the gamma curve, to a result of application of a tone mapping to a low-frequency component of the image; and

determine an enhanced image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image.

2. The system of claim 1 , in which the processing apparatus is configured to determine an output image based on the enhanced image, wherein the determination of the output image comprises the processing apparatus being configured to:

determine gains for respective image portions based on the enhanced image and the gamma curve; and

apply the gains for respective image portions to corresponding image portions of the image to obtain an output image.

3. The system of claim 1 , in which the determination of the low-frequency component perceptual domain image comprises the processing apparatus being configured to:

apply a bilateral filter to the image to obtain the low-frequency component of the image.

4. The system of claim 3 , in which the processing apparatus is configured to:

determine a reduced resolution image based on the image that is at a lower resolution than the image; and

in which applying the bilateral filter comprises processing pixels of the reduced resolution image as candidates.

5. The system of claim 3 , in which the application of the bilateral filter comprises the processing apparatus being configured to:

subsample candidates within a range of distances from a kernel center.

6. The system of claim 3 , in which the application of the bilateral filter comprises the processing apparatus being configured to:

subsample candidates at a first subsampling factor within a first range of distances from a kernel center; and

subsample candidates at a second subsampling factor within a second range of distances from the kernel center.

7. The system of claim 1 , in which the image sensor is attached to the processing apparatus.

8. A method comprising:

receiving an image;

determining a perceptual domain image based on the image and a gamma curve that models human perception of contrast;

determining a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image;

determining an enhanced image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image; and

storing, displaying, or transmitting an output image based on the enhanced image, in which determining the output image comprises determining gains for respective image portions based on the enhanced image and the gamma curve and applying the gains for respective image portions to corresponding image portions of the image.

9. The method of claim 8 , in which determining the low-frequency component perceptual domain image comprises applying a transformation, based on the gamma curve, to a result of applying a tone mapping to a low-frequency component of the image.

10. The method of claim 8 , in which determining the low-frequency component perceptual domain image comprises:

applying a bilateral filter to the image to obtain a low-frequency component image; and

applying a transformation, based on the gamma curve, to a result of applying a tone mapping to the low-frequency component of the image.

11. The method of claim 10 , comprising:

determine a reduced resolution image based on the image that is at a lower resolution than the image; and

in which applying the bilateral filter comprises processing pixels of the reduced resolution image as candidates.

12. The method of claim 10 , in which applying the bilateral filter comprises:

subsampling candidates within a range of distances from a kernel center.

13. The method of claim 10 , in which applying the bilateral filter comprises:

subsampling candidates at a first subsampling factor within a first range of distances from a kernel center; and

subsampling candidates at a second subsampling factor within a second range of distances from the kernel center.

14. A method comprising:

receiving an image from an image sensor;

applying a bilateral filter to the image to obtain a low-frequency component image and a high-frequency component image, wherein applying the bilateral filter includes subsampling candidates within a range of distances from a kernel center;

determining an enhanced image based on a weighted sum of the low-frequency component image and the high-frequency component image, where the high-frequency component image is weighted more than the low-frequency component image; and

storing, displaying, or transmitting an output image based on the enhanced image.

15. The method of claim 14 , in which applying the bilateral filter comprises:

subsampling candidates at a first subsampling factor within a first range of distances from a kernel center; and

subsampling candidates at a second subsampling factor within a second range of distances from the kernel center.

16. The method of claim 14 , in which the subsampling of candidates is implemented with a sparse kernel.

17. The method of claim 14 , in which applying the bilateral filter comprises:

determining a reduced resolution image based on the image that is at a lower resolution than the image; and

processing pixels of the reduced resolution image as candidates.

18. The method of claim 14 , comprising:

determining a perceptual domain image based on the enhanced image and a gamma curve that models human perception of contrast;

determining a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image, in which determining the low-frequency component perceptual domain image comprises applying a transformation, based on the gamma curve, to a result of applying a tone mapping to the low-frequency component image;

determining an enhanced perceptual domain image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image; and

wherein the output image is based on the enhanced perceptual domain image.

19. The method of claim 14 , comprising:

determining a perceptual domain image based on the enhanced image and a gamma curve that models human perception of contrast;

determining a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image, in which determining the low-frequency component perceptual domain image comprises applying the bilateral filter to the perceptual domain image to obtain the low-frequency component perceptual domain image;

determining an enhanced perceptual domain image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image; and

wherein the output image is based on the enhanced perceptual domain image.

20. The method of claim 14 , in which determining the output image comprises:

determining gains for respective image portions based on the enhanced image and the gamma curve; and

applying the gains for respective image portions to corresponding image portions of the image.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 23, 2021
From: GUERIN, GUILLAUME MATTHIEU; REGIMBEAU, ANTOINE; VEIT, THOMAS NICOLAS EMMANUEL; DOUADY-PLEVEN, BRUNO CÉSAR; SUDRET, VIOLAINE MARIE MONG-LAN
To: GOPRO, INC.
Reel/Frame 056637/0572 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
Continuity (3)
Continuation 16368974 · Mar 29, 2019
Continuation 15690772 · Aug 30, 2017
Related Publication 20200177786A1 · Jun 4, 2020
Cited By (1)
US 12,489,984