IP Library Granted Patent US 9,659,349
Granted Patent B2
US 9,659,349 · App. 15/081,677 · Granted May 23, 2017

Color filter array scaler

Inventors: Bruno Cesar Douady-Pleven (Gif-sur-Yvette, FR); Michael Serge André Kraak (Cachan, FR); Guillaume Matthieu Guerin (Paris, FR); Thomas Nicolas Emmanuel Veit (Meudon, FR)
Assignee: GoPro, Inc.
G06T3/4015G06T5/004G06T5/009G06T5/40H04N5/213H04N5/217H04N9/646H04N9/77H04N9/735H04N2209/046
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,659,349
App. No.
15/081,677
Granted
May 23, 2017
Kind
B2
Abstract

A system identifies a scaling position in a captured image, and identifies red subpixels adjacent to the scaling position. The system computes a scaled red subpixel for the scaling position based on the identified red subpixels according to constraints. The system further computes a scaled blue subpixel based on identified adjacent blue subpixels, according to constraints, and computes a scaled green subpixel position based on Gr and Gb subpixels adjacent to the scaling position according to certain constraints. The system then generates a scaled image representative of the captured image, the scaled image including at least the scaled red subpixel value, the scaled blue subpixel value, and the scaled green subpixel value.

Claims (66)

1. A method for generating a scaled image captured by a camera comprising:

identifying a scaling position in a captured image, the captured image comprising an array of subpixels;

identifying a first set of red subpixels within the captured image adjacent to the identified scaling position;

computing a scaled red subpixel value at the scaling position based on values of the first set of red subpixels and the distance of each of the first set of red subpixels to the identified scaling position, the scaled red subpixel value computed such that an amount of noise and an amount of blur corresponding to the scaled red subpixel value are within a threshold of an amount of noise and amount of blur, respectively, corresponding to the first set of red subpixels;

identifying a second set of blue subpixels within the captured image adjacent to the identified scaling position;

computing a scaled blue subpixel value at the scaling position based on values of the second set of blue subpixels and the distance of each of the first set of blue subpixels to the identified scaling position, the scaled blue subpixel value computed such that an amount of noise and an amount of blur corresponding to the scaled blue subpixel value are within a threshold of an amount of noise and amount of blur, respectively, corresponding to the second set of blue subpixels;

identifying a third set of green subpixels within the captured image adjacent to the identified scaling position, the third set of green subpixels including Gr and Gb green subpixels;

computing a scaled green subpixel value at the scaling position based on values of the third set of green subpixels and the distance of each of the third set of green subpixels to the identified scaling position, the scaled green subpixel value computed such that an amount of noise and an amount of blur corresponding to the scaled green subpixel value are within a threshold of an amount of noise and amount of blur, respectively, corresponding to the third set of green subpixels, and such that an imbalance in values between the Gr and Gb green subpixels of the third set of green subpixels is within a threshold of imbalance; and

generating a scaled image representative of the captured image, the scaled image including at least the scaled red subpixel value, the scaled blue subpixel value, and the scaled green subpixel value.

2. The method of claim 1 , wherein identifying a scaling position in a captured image further comprises:

identifying a point at any location on the plane of the captured image; and

selecting the identified point as the scaling position.

3. The method of claim 1 , wherein identifying the first set of red subpixels further comprises:

identifying as the first set of red subpixels a grid of three by three red subpixels including the scaling position within the grid of three by three red subpixels.

4. The method of claim 3 , wherein computing the scaled red subpixel value at the scaling position further comprises:

associating three row coefficients with each row of the grid of three by three red subpixels;

associating three column coefficients with each column of the grid of red subpixels;

associating, for each red subpixel in the grid of red subpixels, a pair of coefficients including a row coefficient and a column coefficient corresponding to the row and column position of the red subpixel in the grid of red subpixels;

determining the value of the three row coefficients and the three column coefficients by solving for each coefficient based on a set of first constraints; and

determining the value of the scaled red subpixel at the scaling position by summing the values of each red subpixel in the first set of red subpixels modified by the corresponding pair of coefficients.

5. The method of claim 1 , wherein identifying the second set of blue subpixels further comprises:

identifying as the second set of blue subpixels a grid of three by three blue subpixels including the scaling position within the grid of three by three blue subpixels.

6. The method of claim 1 , wherein identifying a third set of green subpixels further comprises:

identifying as the third set of green subpixels a set of eight green subpixels within a grid of four by four subpixels surrounding the scaling position, the scaling position bounded by the two centrally located green subpixels within the grid of four by four subpixels.

7. The method of claim 6 , wherein computing the scaled green subpixel value at the scaling position further comprises:

associating four row coefficients with each row of the grid of four by four subpixels (the grid of subpixels);

associating four column coefficients with each column of the grid of subpixels;

associating, for each green subpixel in the grid of subpixels, a pair of coefficients including a row coefficient and a column coefficient corresponding to the row and column position of the green subpixel in the grid of subpixels;

determining the value of the four row coefficients and the four column coefficients by solving for each coefficient based on a set of third constraints; and

determining the value of the scaled green subpixel at the scaling position by summing the values of each green subpixel in the third set of green subpixels modified by the corresponding pair of coefficients.

8. The method of claim 1 , wherein generating a scaled image representative of the captured image further comprises:

computing additional red, green, and blue scaled subpixel values for a plurality of scaling positions of the captured image; and

generating a scaled image including a plurality of full color values computed at each of the plurality of scaling positions computed using the component red, green, and blue scaled subpixel values.

9. A specially-configured hardware system for generating a scaled image configured to:

identify a scaling position in a captured image, the captured image comprising an array of subpixels;

identify a first set of red subpixels within the captured image adjacent to the identified scaling position;

compute a scaled red subpixel value at the scaling position based on values of the first set of red subpixels and the distance of each of the first set of red subpixels to the identified scaling position, the scaled red subpixel value computed such that an amount of noise and an amount of blur corresponding to the scaled red subpixel value are within a threshold of an amount of noise and amount of blur, respectively, corresponding to the first set of red subpixels;

identify a second set of blue subpixels within the captured image adjacent to the identified scaling position;

compute a scaled blue subpixel value at the scaling position based on values of the second set of blue subpixels and the distance of each of the first set of blue subpixels to the identified scaling position, the scaled blue subpixel value computed such that an amount of noise and an amount of blur corresponding to the scaled blue subpixel value are within a threshold of an amount of noise and amount of blur, respectively, corresponding to the second set of blue subpixels;

identify a third set of green subpixels within the captured image adjacent to the identified scaling position, the third set of green subpixels including Gr and Gb green subpixels;

compute a scaled green subpixel value at the scaling position based on values of the third set of green subpixels and the distance of each of the third set of green subpixels to the identified scaling position, the scaled green subpixel value computed such that an amount of noise and an amount of blur corresponding to the scaled green subpixel value are within a threshold of an amount of noise and amount of blur, respectively, corresponding to the third set of green subpixels, and such that an imbalance in values between the Gr and Gb green subpixels of the third set of green subpixels is within a threshold of imbalance; and

generate a scaled image representative of the captured image, the scaled image including at least the scaled red subpixel value, the scaled blue subpixel value, and the scaled green subpixel value.

10. The specially-configured hardware system of claim 9 , further configured to:

identify a point at any location on the plane of the captured image; and

select the identified point as the scaling position.

11. The specially-configured hardware system of claim 9 , further configured to:

identify as the first set of red subpixels a grid of three by three red subpixels including the scaling position within the grid of three by three red subpixels.

12. The specially-configured hardware system of claim 11 , further configured to:

associate three row coefficients with each row of the grid of three by three red subpixels;

associate three column coefficients with each column of the grid of red subpixels;

associate, for each red subpixel in the grid of red subpixels, a pair of coefficients including a row coefficient and a column coefficient corresponding to the row and column position of the red subpixel in the grid of red subpixels;

determine the value of the three row coefficients and the three column coefficients by solving for each coefficient based on a set of first constraints; and

determine the value of the scaled red subpixel at the scaling position by summing the values of each red subpixel in the first set of red subpixels modified by the corresponding pair of coefficients.

13. The specially-configured hardware system of claim 9 , further configured to:

identify as the second set of blue subpixels a grid of three by three blue subpixels including the scaling position within the grid of three by three blue subpixels.

14. The specially-configured hardware system of claim 9 , further configured to:

identify as the third set of green subpixels a set of eight green subpixels within a grid of four by four subpixels surrounding the scaling position, the scaling position bounded by the two centrally located green subpixels within the grid of four by four subpixels.

15. The specially-configured hardware system of claim 14 , further configured to:

associate four row coefficients with each row of the grid of four by four subpixels (the grid of subpixels);

associate four column coefficients with each column of the grid of subpixels;

associate, for each green subpixel in the grid of subpixels, a pair of coefficients including a row coefficient and a column coefficient corresponding to the row and column position of the green subpixel in the grid of subpixels;

determine the value of the four row coefficients and the four column coefficients by solving for each coefficient based on a set of third constraints; and

determine the value of the scaled green subpixel at the scaling position by summing the values of each green subpixel in the third set of green subpixels modified by the corresponding pair of coefficients.

16. The specially-configured hardware system of claim 9 , further configured to:

compute additional red, green, and blue scaled subpixel values for a plurality of scaling positions of the captured image; and

generate a scaled image including a plurality of full color values computed at each of the plurality of scaling positions computed using the component red, green, and blue scaled subpixel values.

Assignments (5)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
RELEASE OF PATENT SECURITY INTEREST Recorded Jan 25, 2021
From: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
To: GOPRO, INC.
Reel/Frame 055106/0434 →
SECURITY AGREEMENT Recorded Aug 29, 2016
From: GOPRO, INC.
To: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 039851/0611 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 13, 2016
From: DOUADY-PLEVEN, BRUNO CESAR; KRAAK, MICHAEL SERGE ANDRE; GUERIN, GUILLAUME MATTHIEU; VEIT, THOMAS NICOLAS EMMANUEL
To: GOPRO, INC.
Reel/Frame 038427/0642 →
Continuity (3)
Provisional Application 62174786 · Jun 12, 2015
Provisional Application 62268687 · Dec 17, 2015
Related Publication 20160364841A1 · Dec 15, 2016