IP Library Granted Patent US 9,754,349
Granted Patent B2
US 9,754,349 · App. 15/081,680 · Granted Sep 5, 2017

Prevention of highlight clipping

Inventors: Bruno Cesar Douady-Pleven (Gif-sur-Yvette, FR); Michael Serge André Kraak (Cachan, FR); Guillaume Matthieu Guerin (Paris, FR); Thomas Nicolas Emmanuel Veit (Meudon, FR)
Assignee: GoPro, Inc.
G06T3/4015G06T5/004G06T5/009G06T5/40H04N5/213H04N5/217H04N5/2355H04N5/357H04N9/646H04N9/77H04N9/735H04N2209/046
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,754,349
App. No.
15/081,680
Granted
Sep 5, 2017
Kind
B2
Abstract

A system determines for each color channel of each portion of the image, a corresponding adjustment value to apply to the color channel to correct for a color irregularity. The system determines a corrected adjustment value based on a difference between twice the pixel value and the maximum saturation value. If the adjustment value as applied is larger than the corrected adjustment value, the system applies the adjustment value to the corresponding color channel of the image portion to produce the adjusted color channel. Otherwise, the system applies the corrected adjustment to the corresponding color channel of the image portion to produce an adjusted color channel. The system generates a modified image based on the adjusted color channel.

Claims (56)

1. A method for processing an image captured by a camera comprising:

accessing an image from an image sensor, the image comprising image portions, at least some of the image portions comprising color channels;

determining, for a color channel of an image portion of the image, an adjustment value to apply to the color channel to correct for a color irregularity, the color channel comprising pixels, each pixel having a pixel value between zero and a maximum saturation value;

determining, for a pixel of the color channel, a corrected adjustment value based on a difference between twice the pixel value of the pixel and the maximum saturation value;

in response to a determination that the adjustment value as applied to the pixel of the color channel is larger than the corrected adjustment value, applying the adjustment value to the pixel to produce an adjusted color channel;

in response to a determination that the adjustment value as applied to the pixel of the color channel is smaller than the corrected adjustment value, applying the corrected adjustment value to the pixel to produce the adjusted color channel; and

generating a modified image based on the adjusted color channel.

2. The method of claim 1 , wherein the color irregularity is a white balance error, and the adjustment value is to be applied uniformly to the image portions of the image.

3. The method of claim 1 , wherein the color irregularity is a lens shading error, and the adjustment value is to be applied to image portions of the image affected by the lens shading error.

4. The method of claim 1 , wherein the adjustment value to be applied is applied for each color channel of one or more of the image portions.

5. The method of claim 1 , wherein the image has a luminance, the method further comprising prior to the determining the adjustment value:

identifying a gain value for the image to increase the luminance of the image;

applying the gain value to each color channel of the image;

determining that one or more pixel values of one or more of the color channels is to be decreased to correct for the color irregularity; and

determining that at least one pixel value of one of the color channels has reached a maximum saturation value due to the applying the gain value.

6. The method of claim 1 , wherein the image has a luminance, the method further comprising:

identifying a gain value for the image to increase the luminance of the image;

applying the gain value to each color channel of the image by:

determining a continuously variable adjusted gain value based on the gain value and dependent upon an input pixel value of each color channel of the image, such that first input pixel values that are nearer a zero value have an adjusted gain value nearing the gain value, and second input pixel values that are nearer the maximum saturation value have an adjusted gain value that when applied to the pixel value does not cause the pixel value to reach the maximum saturation value; and

applying the continuously variable adjusted gain value to the pixel values of one or more of the color channels of the image.

7. The method of claim 1 , wherein the applying the adjustment value comprises:

determining one or more adjusted pixel values of the adjusted color channel by multiplying, for one or more pixels of the color channel, respective pixel values by the adjustment value.

8. The method of claim 1 , wherein the applying the adjustment value comprises:

determining one or more adjusted pixel values of the adjusted color channel by multiplying, for one or more pixels of the color channel, respective pixel values by the corrected adjustment value, the corrected adjustment value of a pixel being equal to twice the pixel value of the pixel subtracted by the maximum saturation value.

9. The method of claim 1 , wherein the applying the corrected adjustment value prevents a presentation of an unnatural color when one or more color channels of one or more of the image portions is desaturated to correct for the color irregularity.

10. A specially-configured hardware system configured to:

access an image from an image sensor, the image comprising image portions, at least some of the image portions comprising color channels;

determine, for a color channel of an image portion of the image, an adjustment value to apply to the color channel to correct for a color irregularity, the color channel comprising pixels, each pixel having a pixel value between zero and a maximum saturation value;

determine, for a pixel of the color channel, a corrected adjustment value based on a difference between twice the pixel value of the pixel and the maximum saturation value;

in response to a determination that the adjustment value as applied to the pixel of the color channel is larger than the corrected adjustment value, apply the adjustment value to the pixel to produce an adjusted color channel;

in response to a determination that the adjustment value as applied to the pixel of the color channel is smaller than the corrected adjustment value, apply the corrected adjustment value to the pixel to produce an adjusted color channel; and

generate a modified image based on the adjusted color channel.

11. The specially-configured hardware system of claim 10 , wherein the color irregularity is a white balance error, and the adjustment value is to be applied uniformly to the image portions of the image.

12. The specially-configured hardware system of claim 10 , wherein the color irregularity is a lens shading error, and the adjustment value is to be applied to image portions of the image affected by the lens shading error.

13. The specially-configured hardware system of claim 10 , wherein the adjustment value to be applied is applied for each color channel of one or more of the image portions.

14. The specially-configured hardware system of claim 10 , wherein the image has a luminance, further configured to:

identify a gain value for the image to increase the luminance of the image;

apply the gain value to each color channel of the image;

determine that one or more pixel values of one or more of the color channels is to be decreased to correct for the color irregularity; and

determine that at least one pixel value of one of the color channels has reached a maximum saturation value due to the applying the gain value.

15. The specially-configured hardware system of claim 10 , wherein the image has a luminance, further configured to:

identify a gain value for the image to increase the luminance of the image;

apply the gain value to each color channel of the image by:

determine a continuously variable adjusted gain value based on the gain value and dependent upon an input pixel value of each color channel of the image, such that first input pixel values that are nearer a zero value have an adjusted gain value nearing the gain value, and second input pixel values that are nearer the maximum saturation value have an adjusted gain value that when applied to the pixel value does not cause the pixel value to reach the maximum saturation value; and

apply the continuously variable adjusted gain value to the pixel values of one or more of the color channels of the image.

16. The specially-configured hardware system of claim 10 , further configured to:

determine one or more adjusted pixel values of the adjusted color channel by multiplying, for one or more pixels of the color channel, respective pixel values by the adjustment value.

17. The specially-configured hardware system of claim 10 , further configured to:

determine one or more adjusted pixel values of the adjusted color channel by multiplying, for one or more pixels of the color channel, respective pixel values by the corrected adjustment value, the corrected adjustment value of a pixel being equal to twice the pixel value of the pixel subtracted by the maximum saturation value.

18. The specially-configured hardware system of claim 10 , wherein the application of the corrected adjustment value prevents a presentation of an unnatural color when one or more color channels of one or more of the image portions is desaturated to correct for the color irregularity.

19. A method for processing an image captured by a camera comprising:

accessing an image from an image sensor, the image comprising image portions, at least some of the image portions comprising color channels;

determining, for a color channel of an image portion of the image, an adjustment value to apply to the color channel to correct for a color irregularity, the color channel comprising pixels, each pixel having a pixel value between zero and a maximum saturation value;

determining, for a pixel of the color channel, a corrected adjustment value based on a difference between twice the pixel value of the pixel and the maximum saturation value;

applying a larger of the adjustment value as applied to the pixel and the corrected adjustment value to produce an adjusted color channel; and

generating a modified image based on the adjusted color channel.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY AGREEMENT Recorded Aug 29, 2016
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 039851/0611 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 13, 2016
From: DOUADY-PLEVEN, BRUNO CESAR; KRAAK, MICHAEL SERGE ANDRE; GUERIN, GUILLAUME MATTHIEU; VEIT, THOMAS NICOLAS EMMANUEL
To: GOPRO, INC.
Reel/Frame 038271/0315 →
Continuity (3)
Provisional Application 62174786 · Jun 12, 2015
Provisional Application 62268687 · Dec 17, 2015
Related Publication 20160366386A1 · Dec 15, 2016