IP Library Granted Patent US 11,463,630
Granted Patent B2
US 11,463,630 · App. 16/931,286 · Granted Oct 4, 2022

Systems and methods for generating a high-dynamic range (HDR) pixel stream

Inventors: William Rivard (Menlo Park, CA); Adam Feder (Mountain View, CA); Brian Kindle (Sunnyvale, CA)
Assignee: DUELIGHT LLC
H04N5/2355G06T5/50H04N5/2352H04N5/23254H04N5/243G06T2207/20208G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,463,630
App. No.
16/931,286
Granted
Oct 4, 2022
Kind
B2
Abstract

A system, method, and computer program product are provided for performing high dynamic range mixing on a pixel stream. In use, a pixel stream including at least two exposures per pixel from a plurality of pixels of an image sensor is received. A first pixel attribute of a first exposure of the pixel is identified, and a second pixel attribute of a second exposure of the pixel is identified. A first high dynamic range (HDR) pixel is generated by performing HDR mixing on the first exposure of the pixel and the second exposure of the pixel, based on the first pixel attribute, and the second pixel attribute. Additionally, an HDR pixel stream is generated comprising at least the first HDR pixel, and the HDR pixel stream is outputted.

Claims (41)

1. An apparatus, comprising:

circuitry for:

receiving a pixel stream including at least two exposures per pixel from a plurality of pixels of an image sensor, wherein receiving the pixel stream includes receiving a first exposure of a pixel, and a second exposure of the pixel;

identifying a first pixel attribute of the first exposure of the pixel;

identifying a second pixel attribute of the second exposure of the pixel;

generating a first high dynamic range (HDR) pixel by performing HDR mixing on the first exposure of the pixel and the second exposure of the pixel, based on both of the first pixel attribute and the second pixel attribute;

generating an HDR pixel stream comprising at least the first HDR pixel;

outputting the HDR pixel stream; and

outputting the first exposure of the pixel within a first pixel stream.

2. The apparatus of claim 1 , wherein the apparatus is operable such that the first pixel attribute includes a first intensity, and the second pixel attribute includes a second intensity.

3. The apparatus of claim 1 , wherein the apparatus is operable such that the first pixel attribute includes a first saturation, and the second pixel attribute includes a second saturation.

4. The apparatus of claim 1 , further comprising outputting the second exposure of the pixel within a second pixel stream.

5. The apparatus of claim 1 , further comprising outputting the first exposure of the pixel and the second exposure of the pixel within a third pixel stream.

6. The apparatus of claim 1 , wherein the first exposure of the pixel is a medium exposure pixel, and the second exposure of the pixel is a dark exposure pixel.

7. The apparatus of claim 1 , wherein the first exposure of the pixel is a medium exposure pixel, and the second exposure of the pixel is a bright exposure pixel.

8. The apparatus of claim 1 , wherein the first exposure of the pixel is a dark exposure pixel, and the second exposure of the pixel is sampled at a bright exposure pixel.

9. The apparatus of claim 1 , wherein the HDR pixel stream is modified by an effects function to generate an output HDR image.

10. The apparatus of claim 9 , wherein the effects function performs at least one of modifying contrast, modifying saturation, and modifying noise.

11. The apparatus of claim 1 , wherein the HDR mixing comprises at least one of local range compression and global range compression.

12. The apparatus of claim 1 , wherein the HDR mixing comprises bilateral filtering.

13. The apparatus of claim 1 , wherein the HDR pixel stream includes a plurality of HDR pixels such that each HDR pixel of the plurality of HDR pixels is associated with a different pixel of the image sensor, and each of the HDR pixels is generated based on the at least two exposures.

14. The apparatus of claim 1 , wherein one or more of the pixel stream and the HDR pixel stream is stored in a memory circuit.

15. The apparatus of claim 1 , the circuitry further for at least one of: identifying a first scalar based on the first pixel attribute and the second pixel attribute, or identifying a second scalar based on the second pixel attribute and a third pixel attribute.

16. The apparatus of claim 15 , the circuitry further for at least one of: blending, based on the first scalar, the first exposure of the pixel and the second exposure of the pixel, or blending, based on the second scalar, the second exposure of the pixel and a third exposure of the pixel.

17. The apparatus of claim 1 , the circuitry further for identifying a scalar based on the first pixel attribute and the second pixel attribute, the scalar being identified based on a polynomial.

18. A computer program comprising computer executable instructions stored on a non-transitory computer readable medium that when that when executed by one or more processors instruct the one or more processors to:

receive a pixel stream including at least two exposures per pixel from a plurality of pixels of an image sensor, wherein receiving the pixel stream includes receiving a first exposure of a pixel, and a second exposure of the pixel;

identify a first pixel attribute of the first exposure of the pixel;

identify a second pixel attribute of the second exposure of the pixel;

generate a first high dynamic range (HDR) pixel by performing HDR mixing on the first exposure of the pixel and the second exposure of the pixel, based on both of the first pixel attribute and the second pixel attribute;

generate an HDR pixel stream comprising at least the first HDR pixel;

output the HDR pixel stream; and

output the first exposure of the pixel within a first pixel stream.

19. A method, comprising:

receiving a pixel stream including at least two exposures per pixel from a plurality of pixels of an image sensor, wherein receiving the pixel stream includes receiving a first exposure of a pixel, and a second exposure of the pixel;

identifying a first pixel attribute of the first exposure of the pixel;

identifying a second pixel attribute of the second exposure of the pixel;

generating a first high dynamic range (HDR) pixel by performing HDR mixing on the first exposure of the pixel and the second exposure of the pixel, based on both of the first pixel attribute and the second pixel attribute;

generating an HDR pixel stream comprising at least the first HDR pixel;

outputting the HDR pixel stream; and

outputting the first exposure of the pixel within a first pixel stream.

Assignments (2)
SECURITY INTEREST Recorded Jul 30, 2021
From: DUELIGHT, LLC
To: ZILKA KOTAB, P.C.
Reel/Frame 057043/0301 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 17, 2020
From: RIVARD, WILLIAM; FEDER, ADAM; KINDLE, BRIAN
To: DUELIGHT LLC
Reel/Frame 053245/0840 →
Continuity (6)
Continuation In Part 16857016 · Apr 23, 2020
Continuation 16519244 · Jul 23, 2019
Continuation 15891251 · Feb 7, 2018
Continuation 14823993 · Aug 11, 2015
Continuation In Part 14536524 · Nov 7, 2014
Related Publication 20200351432A1 · Nov 5, 2020
Cited By (6)
US 12,401,911 US 12,401,912 US 12,418,727 US 12,445,736 US 12,666,159 US 12,707,151