IP Library Granted Patent US 10,284,785
Granted Patent B2
US 10,284,785 · App. 15/690,772 · Granted May 7, 2019

Local tone mapping

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,284,785
App. No.
15/690,772
Granted
May 7, 2019
Kind
B2
Abstract

Systems and methods are disclosed for image signal processing. For example, methods may include receiving an image from an image sensor; applying a filter to the image to obtain a low-frequency component image and a high-frequency component image; determining a first enhanced image based on a weighted sum of the low-frequency component image and the high-frequency component image, where the high-frequency component image is weighted more than the low-frequency component image; determining a second enhanced image based on the first enhanced image and a tone mapping; and storing, displaying, or transmitting an output image based on the second enhanced image.

Claims (73)

1. A system comprising:

an image sensor configured to capture an image; and

a processing apparatus configured to:

receive the image from the image sensor;

apply a bilateral filter to the image to obtain a low-frequency component image and a high-frequency component image;

determine a first enhanced image based on a weighted sum of the low-frequency component image and the high-frequency component image, where the high-frequency component image is weighted more than the low-frequency component image;

determine a second enhanced image based on the first enhanced image and a tone mapping;

determine a perceptual domain image based on the second enhanced image and a gamma curve that models human perception of contrast;

determine a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image;

determine a third enhanced image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image; and

store, display, or transmit an output image based on the third enhanced image.

2. The system of claim 1 , in which determining the second enhanced image based on the first enhanced image and the tone mapping comprises:

applying the tone mapping to the low-frequency component image to obtain gains for respective image portions; and

applying the gains for respective image portions to corresponding image portions of the first enhanced image.

3. The system of claim 1 , in which the processing apparatus is configured to:

determine the tone mapping based on a histogram analysis of image portions of the of the low-frequency component image.

4. The system of claim 1 , in which determining the low-frequency component perceptual domain image comprises applying a transformation, based on the gamma curve, to a result of applying the tone mapping to the low-frequency component image.

5. The system of claim 1 , in which the output image is determined by performing operations comprises:

determining gains for respective image portions based on the third enhanced image and the gamma curve; and

applying the gains for respective image portions to corresponding image portions of the image.

6. The system of claim 1 , in which the processing apparatus is configured to:

determine a reduced resolution image based on the image that is at a lower resolution than the image; and

in which applying the bilateral filter comprises processing pixels of the reduced resolution image as candidates.

7. The system of claim 1 , in which applying the bilateral filter comprises:

subsampling candidates within a range of distances from a kernel center.

8. The system of claim 1 , in which applying the bilateral filter comprises:

subsampling candidates at a first subsampling factor within a first range of distances from a kernel center; and

subsampling candidates at a second subsampling factor within a second range of distances from the kernel center.

9. The system of claim 1 , in which determining the first enhanced image comprises:

checking an underflow condition for an image portion of the first enhanced image; and

where the underflow condition occurs, adding an offset to a corresponding image portion of the low-frequency component image, and subtracting the offset from a corresponding image portion of the high-frequency component image.

10. The system of claim 1 , in which the image sensor is attached to the processing apparatus.

11. A method, comprising:

receiving an image from an image sensor;

applying a filter to the image to obtain a low-frequency component image and a high-frequency component image;

determining a first enhanced image based on a weighted sum of the low-frequency component image and the high-frequency component image, where the high-frequency component image is weighted more than the low-frequency component image;

determining a second enhanced image based on the first enhanced image and a tone mapping;

storing, displaying, or transmitting an output image based on the second enhanced image;

the method further comprising,

determining a perceptual domain image based on the second enhanced image and a gamma curve that models human perception of contrast;

determining a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image;

determining a third enhanced image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image; and

wherein the output image is based on the third enhanced image.

12. The method of claim 11 , in which determining the low-frequency component perceptual domain image comprises applying a transformation, based on the gamma curve, to a result of applying the tone mapping to the low-frequency component image.

13. The method of claim 11 , in which the output image is determined by performing operations comprises:

determining gains for respective image portions based on the third enhanced image and the gamma curve; and

applying the gains for respective image portions to corresponding image portions of the image.

14. A system, comprising:

an image sensor configured to capture an image; and

a processing apparatus configured to:

receive the image from the image sensor;

apply a filter to the image to obtain a low-frequency component image and a high-frequency component image;

apply a non-linear mapping to the low-frequency component image to obtain gains for respective image portions;

apply the gains for respective image portions to corresponding image portions of the image to obtain an enhanced image;

store, display, or transmit an output image based on the enhanced image; and

wherein the processing apparatus is further configured to determine the non-linear mapping based on a histogram analysis of image portions of the of the low-frequency component image.

15. The system of claim 14 , in which the processing apparatus is configured to:

determine a perceptual domain image based on the enhanced image and a gamma curve that models human perception of contrast;

determine a low-frequency component perceptual domain image and a high-frequency component perceptual domain image as components of the perceptual domain image;

determine an enhanced perceptual domain image based on a weighted sum of the low-frequency component perceptual domain image and the high-frequency component perceptual domain image, where the high-frequency component perceptual domain image is weighted more than the low-frequency component perceptual domain image; and

wherein the output image is based on the enhanced perceptual domain image.

16. The system of claim 15 , in which determining the low-frequency component perceptual domain image comprises applying a transformation, based on the gamma curve, to a result of applying the gains for respective image portions to the low-frequency component image.

17. The system of claim 15 , in which the output image is determined by performing operations comprises:

determining gains for respective image portions based on the enhanced perceptual domain image and the gamma curve; and

applying the gains for respective image portions to corresponding image portions of the image.

18. The system of claim 14 , in which the filter is a bilateral filter and applying the filter comprises:

subsampling candidates within a range of distances from a kernel center.

19. The system of claim 14 , in which the filter is a bilateral filter and applying the filter comprises:

subsampling candidates at a first subsampling factor within a first range of distances from a kernel center; and

subsampling candidates at a second subsampling factor within a second range of distances from the kernel center.

20. The method of claim 11 , in which determining the second enhanced image based on the first enhanced image and a tone mapping comprises:

applying the tone mapping to the low-frequency component image to obtain gains for respective image portions; and

applying the gains for respective image portions to corresponding image portions of the first enhanced image.

Assignments (6)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
SECURITY INTEREST Recorded Nov 20, 2017
From: GOPRO, INC.; GOPRO CARE, INC.; GOPRO CARE SERVICES, INC.; WOODMAN LABS CAYMAN, INC.
To: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
Reel/Frame 044180/0430 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 30, 2017
From: GUERIN, GUILLAUME MATTHIEU; REGIMBEAU, ANTOINE; VEIT, THOMAS NICOLAS EMMANUEL; DOUADY-PLEVEN, BRUNO CÉSAR; SUDRET, VIOLAINE MARIE MONG-LAN
To: GOPRO, INC.
Reel/Frame 043448/0951 →
Cited By (1)
US 12,489,984