IP Library Granted Patent US 9,842,381
Granted Patent B2
US 9,842,381 · App. 15/081,682 · Granted Dec 12, 2017

Global tone mapping

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,842,381
App. No.
15/081,682
Granted
Dec 12, 2017
Kind
B2
Abstract

A system accesses an image with each pixel of the image having luminance values each representative of a color component of the pixel. The system generates a first histogram for aggregate luminance values of the image, and accesses a target histogram for the image representative of a desired global image contrast. The system computes a transfer function based on the first histogram and the target histogram such that when the transfer function is applied, a histogram of the modified aggregate luminance values is within a threshold similarity of the target histogram. The system modifies the image by applying the transfer function to the luminance values of the image to produce a tone mapped image, and outputs the modified image.

Claims (27)

1. A method for generating a tone mapped image captured by a camera comprising: accessing an image captured by an image sensor, the accessed image comprising, for each image pixel of the accessed image, a corresponding set of luminance values each representative of a color component of the pixel; generating a first histogram for aggregate luminance values of the image; accessing a target histogram for the image representative of a desired global image contrast; computing a transfer function based on the first histogram and the target histogram such that when the transfer function is applied to the aggregate luminance values of the image to create a set of modified aggregate luminance values, a histogram of the modified aggregate luminance values is within a threshold similarity of the target histogram; modifying the accessed image by applying the transfer function to the set of luminance values corresponding to each pixel of the image to produce a tone mapped image; and outputting the modified image.

2. The method of claim 1 , further comprising: computing, for each pixel of the accessed image, an aggregate luminance value based on a luminance function applied to the set of luminance values corresponding to each color component of the pixel.

3. The method of claim 1 , further comprising: generating a histogram for each color channel of the accessed image; and applying a black point to the image for each color channel based on the generated histograms for each color channel to establish a lower bounds for each color channel.

4. The method of claim 1 , further comprising: generating an exposure correction curve based on a lookup table, the curve compensating for portions of the first histogram of the luminance values that are representative of an underexposure of the image within a corresponding portion of a color channel, the lookup table producing a linear output for brightness values below a threshold and a compressed non-linear output for brightness values above a threshold; applying the exposure correction curve to a luminance value of each pixel in the image to generate exposure corrected luminance values of the image; and transposing the transfer function from a post-gamma curve space to the original linear space of the image.

5. The method of claim 1 , further comprising: adjusting the transfer function to remove above-threshold amplification of noise resulting from an application of the transfer function to the image.

6. The method of claim 5 , further comprising: computing a derivative of the transfer function; adjusting one of a high threshold area and a low threshold area of the derivative of the transfer function such that the high threshold area and the low threshold area are equal; modifying the derivative by removing the high threshold area and the low threshold area from the derivative; and computing a modified transfer function from the modified derivative.

7. The method of claim 1 , further comprising: applying an unsharp mask to portions of the image that include areas of interest in order to increase local contrast.

8. The method of claim 7 , further comprising: determining one or more changed image portions of the modified image that are one of compressed and amplified compared to the corresponding image portion of the accessed image; and modifying these changed image portions to increase the local contrast in these changed image portions.

9. The method of claim 1 , further comprising: in response to determining that the accessed image is uniform by determining that a threshold number of the pixels of the accessed image are within a threshold level of each other, modifying the transfer function to be uniform.

10. The method of claim 1 , further comprising: computing the gain in the linear domain equivalent to the transfer function previously estimated in the non-linear domain.

11. A specially-configured hardware system for generating a tone mapped image configured to: access an image captured by an image sensor, the accessed image comprising, for each image pixel of the accessed image, a corresponding set of luminance values each representative of a color component of the pixel; generate a first histogram for aggregate luminance values of the image; access a target histogram for the image representative of a desired global image contrast; compute a transfer function based on the first histogram and the target histogram such that when the transfer function is applied to the aggregate luminance values of the image to create a set of modified aggregate luminance values, a histogram of the modified aggregate luminance values is within a threshold similarity of the target histogram; modify the accessed image by applying the transfer function to the set of luminance values corresponding to each pixel of the image to produce a tone mapped image; and output the modified image.

12. The specially-configured hardware system of claim 11 , further configured to: compute, for each pixel of the accessed image, an aggregate luminance value based on a luminance function applied to the set of luminance values corresponding to each color component of the pixel.

13. The specially-configured hardware system of claim 11 , further configured to: generate a histogram for each color channel of the accessed image; and apply a black point to the image for each color channel based on the generated histograms for each color channel to establish a lower bounds for each color channel.

14. The specially-configured hardware system of claim 11 , further configured to: generate an exposure correction curve based on a lookup table, the curve compensating for portions of the first histogram of the luminance values that are representative of an underexposure of the image within a corresponding portion of a color channel, the lookup table producing a linear output for brightness values below a threshold and a compressed non-linear output for brightness values above a threshold; apply the exposure correction curve to a luminance value of each pixel in the image to generate exposure corrected luminance values of the image; and transpose the transfer function from a post-gamma curve space to the original linear space of the image.

15. The specially-configured hardware system of claim 11 , further configured to: adjust the transfer function to remove above-threshold amplification of noise resulting from an application of the transfer function to the image.

16. The specially-configured hardware system of claim 15 , further configured to: compute a derivative of the transfer function; adjust one of a high threshold area and a low threshold area of the derivative of the transfer function such that the high threshold area and the low threshold area are equal; modify the derivative by removing the high threshold area and the low threshold area from the derivative; and compute a modified transfer function from the modified derivative.

17. The specially-configured hardware system of claim 11 , further configured to: apply an unsharp mask to portions of the image that include areas of interest in order to increase local contrast.

18. The specially-configured hardware system of claim 17 , further configured to: determine one or more changed image portions of the modified image that are one of compressed and amplified compared to the corresponding image portion of the accessed image; and modify these changed image portions to increase the local contrast in these changed image portions.

19. The specially-configured hardware system of claim 11 , further configured to: in response to a determination that the accessed image is uniform by determining that a threshold number of the pixels of the accessed image are within a threshold level of each other, modify the transfer function to be uniform.

20. A method for generating a tone mapped image captured by a camera comprising:

accessing an image captured by an image sensor, the accessed image comprising, for each image pixel of the accessed image, a corresponding set of luminance values each representative of a color component of the pixel;

accessing an image processing setting that, when applied to the accessed image, produces an image effect;

selecting a target histogram for the image based on the accessed image processing setting;

generating a first histogram for aggregate luminance values of the image;

computing a transfer function based on the first histogram and the target histogram such that when the transfer function is applied to the aggregate luminance values of the image to create a set of modified aggregate luminance values, a histogram of the modified aggregate luminance values is within a threshold similarity of the target histogram;

applying the transfer function to the image to obtain a modified image; and

outputting an output image based on the modified image.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY AGREEMENT Recorded Aug 29, 2016
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 039851/0611 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 13, 2016
From: DOUADY-PLEVEN, BRUNO CESAR; KRAAK, MICHAEL SERGE ANDRE; GUERIN, GUILLAUME MATTHIEU; VEIT, THOMAS NICOLAS EMMANUEL
To: GOPRO, INC.
Reel/Frame 038271/0307 →