IP Library Granted Patent US 11,082,625
Granted Patent B2
US 11,082,625 · App. 16/547,786 · Granted Aug 3, 2021

Imaging systems for generating HDR images and operating methods thereof

Inventors: Hee Kang (Hwaseong-si, KR); Youngjun Song (Seoul, KR); Chanyoung Jang (Pohang-si, KR); Minkyu Sun (Suwon-si, KR); Junghoon Jung (Hwaseong-si, KR)
Assignee: Samsung Electronics Co., Ltd.
H04N5/2355G06T3/4015G06T7/20H04N5/2353H04N5/265H04N9/0455G06T5/007G06T2207/20208
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,082,625
App. No.
16/547,786
Granted
Aug 3, 2021
Kind
B2
Abstract

An imaging system for generating a high dynamic range (HDR) image includes a pixel array that includes first pixels and second pixels, a driver circuit configured to control the first pixels based on a first exposure time and control the second pixels based on a second exposure time, an output circuit configured to output first pixel values of the first pixels and second pixel values of the second pixels from the pixel array, and a processing device configured to generate the HDR image based on a first Bayer image and a second Bayer image, where the first Bayer image is generated based on the first pixel values and the second Bayer image is generated based on the second pixel values and third pixel values of the first Bayer image. The quantity of pixels of the first pixels may be greater than the quantity of pixels of the second pixels.

Claims (63)

1. An imaging system, comprising:

a pixel array including a plurality of first pixels and a plurality of second pixels, the plurality of second pixels being separate from the plurality of first pixels;

driver circuitry configured to

control the plurality of first pixels based on a first exposure time, and

control the plurality of second pixels based on a second exposure time;

output circuitry configured to output first pixel values of the plurality of first pixels and second pixel values of the plurality of second pixels from the pixel array; and

processing circuitry configured to generate a high dynamic range (HDR) image based on a first Bayer image and a second Bayer image, the first Bayer image generated based only on the first pixel values, the second Bayer image generated based on both the second pixel values and third pixel values of the first Bayer image;

wherein a quantity of pixels of the first pixels is greater than a quantity of pixels of the second pixels;

wherein the processing circuitry is further configured to

separate the first pixel values and the second pixel values to establish separated first pixel values and separated second pixel values, respectively, based on exposure time information associated with the pixel array,

generate the first Bayer image based only on the separated first pixel values,

generate the second Bayer image based on both the separated second pixel values and the third pixel values of the first Bayer image,

estimate motion information of an object based on the pixel values and the second pixel values, and

determine a weight of the third pixel values to generate the second Bayer image, based on the estimated motion information.

2. The imaging system of claim 1 , wherein the processing circuitry is configured to generate the first Bayer image based on

interpolating pixel values based on the separated first pixel values to generate an interpolation image, and

converting the interpolation image to a Bayer pattern.

3. The imaging system of claim 1 , wherein the processing circuitry is further configured to, in response to a determination that a saturation area including one or more saturated pixels is present in the first Bayer image, generate pixel values of the second Bayer image based on remaining pixel values of the third pixel values that exclude pixel values of the one or more saturated pixels.

4. The imaging system of claim 1 , further comprising:

controller circuitry configured to

determine an exposure time of the first pixels as the first exposure time, and

determine an exposure time of the second pixels as the second exposure time.

5. The imaging system of claim 4 , wherein the controller circuitry is configured to determine the exposure time of the first pixels and the exposure time of the second pixels based on pixel values output from the output circuitry.

6. The imaging system of claim 1 , further comprising:

a color filter array (CFA) including a plurality of color filters positioned over the pixel array,

wherein at least two adjacent color filters of the plurality of color filters are configured to selectively transmit light having a same color that is one of a red color, a green color, or a blue color.

7. The imaging system of claim 1 , wherein the first pixel values and the second pixel values are obtained from a common frame of an image generated by the pixel array.

8. The imaging system of claim 1 , wherein a resolution of the first Bayer image and a resolution of the second Bayer image are the same as a total quantity of pixels of the pixel array.

9. The imaging system of claim 1 , wherein the first exposure time is longer than the second exposure time.

10. An imaging system, comprising:

an image sensor including a plurality of first pixels and a plurality of second pixels, the plurality of second pixels being separate from the plurality of first pixels, the image sensor configured to generate image data associated with one frame of an image based on the plurality of first pixels being controlled during a first exposure time and the plurality of second pixels being controlled during a second exposure time, the image data including first pixel values of the plurality of first pixels and second pixel values of the plurality of second pixels; and

image signal processing circuitry configured to generate an HDR image based on a first Bayer image and a second Bayer image, the first Bayer image generated based on the first pixel values, the second Bayer image generated based on both the second pixel values and third pixel values of the first Bayer image,

wherein a quantity of pixels of the first pixels is greater than a quantity of pixels of the second pixels,

wherein the image signal processing circuitry is further configured to

separate the first pixel values and the second pixel values to establish separated first pixel values and separated second pixel values, respectively, based on exposure time information associated with the image sensor,

generate the first Bayer image based only on the separated first pixel values,

generate the second Bayer image based on both the separated second pixel values and the third pixel values of the first Bayer image,

estimate motion information of an object on the first pixel values and the second pixel values, and

determine a weight of the third pixel values to generate the second Bayer image, based on the estimated motion information.

11. The imaging system of claim 10 , wherein the image signal processing circuitry is configured to combine the first Bayer image and the second Bayer image based on the motion information, to generate the HDR image.

12. An operating method of an imaging system, the imaging system including an image sensor including a plurality of first pixels and a plurality of second pixels, the method comprising:

generating image data associated with one frame of an image based on the plurality of first pixels being controlled during a first exposure time and the plurality of second pixels being controlled during a second exposure time, the image data including first pixel values of the plurality of first pixels and second pixel values of the plurality of second pixels;

separating the first pixel values and the second pixel values to establish separated first pixel values and separated second pixel values, respectively, based on exposure time information associated with the image sensor;

generating a first Bayer image based only on the separated first pixel values of the first pixels of the image data;

generating a second Bayer image based on both the separated second pixel values of the second pixels of the image data and third pixel values of the first Bayer generating an HDR image based on the first Bayer image and the second Bayer image; and

estimating motion information of an object on the one frame based on the image data,

wherein a quantity of pixels of the first pixels is greater than a quantity of pixels of the second pixels,

wherein the plurality of second pixels are separate from the plurality of first pixels,

wherein the generating the second Bayer image includes

determining a first weight of the second pixel values and a second weight of the third pixel values based on the estimated motion information, and

generating pixel values of the second Bayer image based on the first weight and the second weight.

13. The method of claim 12 , wherein the generating the first Bayer image includes

interpolating pixel values based on the first pixel values to generate an interpolation image; and

converting the interpolation image to a Bayer pattern.

14. The method of claim 12 , wherein the generating the HDR image includes

combining the first Bayer image and the second Bayer image based on the motion information.

15. The method of claim 12 , further comprising:

generating, by a motion detection sensor, sensing information associated with the object,

wherein the motion information is estimated based on the image data and the sensing information.

16. The method of claim 12 , further comprising:

detecting a saturation area, which includes one or more saturated pixels, of the first Bayer image,

wherein the generating the second Bayer image includes

generating pixel values of the second Bayer image based on remaining pixel values of the third pixel values that exclude pixel values of the one or more saturated pixels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2019
From: KANG, HEE; SONG, YOUNGJUN; JANG, CHANYOUNG; SUN, MINKYU; JUNG, JUNGHOON
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 050143/0123 →
Priority Claims (1)
KR 10-2019-0006857 · Jan 18, 2019 · national
Continuity (1)
Related Publication 20200236273A1 · Jul 23, 2020
Cited By (2)
US 12,536,615 US 12,731,217