IP Library Granted Patent US 10,692,196
Granted Patent B2
US 10,692,196 · App. 15/955,855 · Granted Jun 23, 2020

Color correction integrations for global tone mapping

Inventors: Guillaume Matthieu Guérin (Paris, FR); Giuseppe Moschetti (Hauts-de-Seine, FR)
Assignee: GoPro, Inc.
G06T5/009H04N9/646H04N9/67H04N9/68G06T2207/10024G06T2207/20208
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,692,196
App. No.
15/955,855
Granted
Jun 23, 2020
Kind
B2
Abstract

Color correction is integrated within global tone mapping operations to modify an image captured using an image capture device. New luminance values are determined for the pixels of the image by performing global tone mapping against those pixels using one or more sets of color correction values, which are applied against respective luminance values and color components of those pixels. The sets of color correction values are identified within 3×3 color correction matrices. A gain curve for modifying contrast values of the image is determined based on at least one of the new luminance values. A modified image is then generated by modifying the contrast values according to the gain curve, for example, by applying a gamma curve against the contrast values using data stored in a three-dimensional lookup table.

Claims (70)

1. An image capture apparatus, comprising:

an image capture device configured to capture an image, the image including multiple pixels, a given pixel being characterized by color components and a luminance value associated with the color components;

a memory; and

a processor configured to execute instructions stored in the memory to:

receive the image from the image capture device;

for the given pixel of the pixels:

identify a color correction matrix including color correction values for modifying luminance coefficients of the given pixel, the luminance coefficients used to determine the luminance value associated with the color components of the given pixel;

determine new luminance coefficients for the given pixel by modifying the luminance coefficients of the given pixel according to individual ones of the color correction values; and

determine a new luminance value for the given pixel by modifying magnitudes of the color components of the given pixel according to the new luminance coefficients;

determine a gain curve for modifying contrast values of the image based on new luminance values determined for at least some of the pixels; and

modify the image by modifying the contrast values according to the gain curve.

2. The image capture apparatus of claim 1 , wherein the luminance coefficients of the given pixel include a first luminance coefficient associated with a first color component of the given pixel, a second luminance coefficient associated with a second color component of the given pixel, and a third luminance coefficient associated with a third color component of the given pixel, and

wherein determining the new luminance coefficients for the given pixel by modifying the luminance coefficients of the given pixel according to the individual ones of the color correction values comprises:

determining a first new luminance coefficient by modifying the luminance coefficients according to values in a first column of the color correction matrix;

determining a second new luminance coefficient by modifying the luminance coefficients according to values in a second column of the color correction matrix; and

determining a third new luminance coefficient by modifying the luminance coefficients according to values in a third column of the color correction matrix.

3. The image capture apparatus of claim 2 , wherein determining the new luminance value for the given pixel by modifying the magnitudes of the color components of the given pixel according to the new luminance coefficients comprises:

determining a first value by modifying a magnitude of the first color component according to the first luminance coefficient;

determining a second value by modifying a magnitude of the second color component according to the second luminance coefficient;

determining a third value by modifying a magnitude of the third color component according to the third luminance coefficient; and

determining the new luminance value by combining the first value, the second value, and the third value.

4. The image capture apparatus of claim 1 , wherein the color correction matrix used for the given pixel is a 3×3 matrix.

5. The image capture apparatus of claim 1 , wherein modifying the image by modifying the contrast values according to the gain curve comprises:

modifying the contrast values according to a gamma curve using data stored in a three-dimensional lookup table.

6. The image capture apparatus of claim 1 , wherein the new luminance values are determined while the image remains in a linear signal.

7. A method for modifying an image captured using an image capture device, the method comprising:

receiving the image from the image capture device, the image including multiple pixels, a given pixel being characterized by color components and a luminance value associated with the color components;

determining new luminance values for the pixels by performing global tone mapping against the pixels using color correction values applied against respective luminance values and color components of the pixels;

determining a gain curve for modifying contrast values of the image based on at least one of the new luminance values; and

modifying the image by modifying the contrast values according to the gain curve.

8. The method of claim 7 , wherein determining the new luminance values for the pixels by performing the global tone mapping against the pixels using the color correction values applied against the respective luminance values and color components of the pixels comprises:

identifying a color correction matrix for modifying luminance coefficients of the given pixel, the color correction matrix including the color correction values, the luminance coefficients of the given pixel used to determine the luminance value associated with the color components of the given pixel;

determining new luminance coefficients for the given pixel by modifying the luminance coefficients according to individual ones of the color correction values; and

determining the new luminance value for the given pixel by modifying magnitudes of the color components of the given pixel according to the new luminance coefficients.

9. The method of claim 8 , wherein the luminance coefficients of the given pixel include a first luminance coefficient associated with a first color component of the given pixel, a second luminance coefficient associated with a second color component of the given pixel, and a third luminance coefficient associated with a third color component of the given pixel, and

wherein determining the new luminance coefficients for the given pixel by modifying the luminance coefficients according to the individual ones of the color correction values comprises:

determining a first new luminance coefficient by modifying the luminance coefficients according to values in a first column of the color correction matrix;

determining a second new luminance coefficient by modifying the luminance coefficients according to values in a second column of the color correction matrix; and

determining a third new luminance coefficient by modifying the luminance coefficients according to values in a third column of the color correction matrix.

10. The method of claim 9 , wherein determining the new luminance value for the given pixel by modifying the magnitudes of the color components of the given pixel according to the new luminance coefficients comprises:

determining a first value by modifying a magnitude of the first color component according to the first luminance coefficient;

determining a second value by modifying a magnitude of the second color component according to the second luminance coefficient;

determining a third value by modifying a magnitude of the third color component according to the third luminance coefficient; and

determining the new luminance value by combining the first value, the second value, and the third value.

11. The method of claim 8 , wherein the color correction matrix is a 3×3 matrix.

12. The method of claim 7 , wherein modifying the image by modifying the contrast values according to the gain curve comprises:

modifying the contrast values according to a gamma curve using data stored in a three-dimensional lookup table.

13. The method of claim 7 , wherein the new luminance values are determined while the image remains in a linear signal.

14. An image processor, comprising:

an image modification processing unit configured to:

receive an image, the image including multiple pixels, a given pixel being characterized by color components and luminance coefficients;

identify a color correction matrix including color correction values for modifying the luminance coefficients of the given pixel;

determine new luminance coefficients for the given pixel by modifying the luminance coefficients of the given pixel according to individual ones of the color correction values; and

modify the image using a new luminance value for the given pixel, the new luminance value determined by modifying magnitudes of the color components of the given pixel according to the new luminance coefficients.

15. The image processor of claim 14 , wherein the luminance coefficients include a first luminance coefficient associated with a first color component of the given pixel, a second luminance coefficient associated with a second color component of the given pixel, and a third luminance coefficient associated with a third color component of the given pixel, and

wherein the image modification processing unit is configured to determine the new luminance coefficients by modifying the luminance coefficients of the given pixel according to the individual ones of the color correction values by:

determining a first new luminance coefficient by modifying the luminance coefficients according to values in a first column of the color correction matrix;

determining a second new luminance coefficient by modifying the luminance coefficients according to values in a second column of the color correction matrix; and

determining a third new luminance coefficient by modifying the luminance coefficients according to values in a third column of the color correction matrix.

16. The image processor of claim 15 , wherein the image modification processing unit is configured to determine the new luminance value by:

determining a first value by modifying a magnitude of the first color component according to the first luminance coefficient;

determining a second value by modifying a magnitude of the second color component according to the second luminance coefficient;

determining a third value by modifying a magnitude of the third color component according to the third luminance coefficient; and

determining the new luminance value by combining the first value, the second value, and the third value.

17. The image processor of claim 14 , wherein the image modification processing unit is configured to modify the image using the new luminance value for the given pixel by:

determining a gain curve for modifying contrast values of the image based on the new luminance value; and

modifying the contrast values according to a gamma curve using data stored in a three-dimensional lookup table.

18. The image processor of claim 14 , wherein the image modification processing unit uses the luminance coefficients to determine a luminance value associated with the color components of the given pixel.

19. The image processor of claim 14 , wherein the image modification processing unit determines new luminance values for at least some of the pixels of the image.

20. The image processor of claim 19 , wherein the image modification processing unit determines the new luminance values while the image remains in a linear signal.

Assignments (6)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY INTEREST Recorded Oct 19, 2020
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 054113/0594 →
SECURITY INTEREST Recorded Sep 5, 2018
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 047016/0417 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 18, 2018
From: GUÉRIN, GUILLAUME MATTHIEU; MOSCHETTI, GIUSEPPE
To: GOPRO, INC.
Reel/Frame 045575/0323 →