IP Library › Granted Patent US 11,570,384
Granted Patent B2
US 11,570,384 · App. 17/011,227 · Granted Jan 31, 2023

Image sensor employing varied intra-frame analog binning

Inventors: Shahaf Duenyas (Yavne, IL); Yoel Yaffe (Modiin, IL); Guy Horowitz (Kfar Saba, IL); Amit Eisenberg (Kiryat Ono, IL); Shy Hamami (Ganei-Tikva, IL); Oded Monzon (Modiin, IL); Gal Bitan (Petach Tikva, IL); Yoav Piepsh (Holon, IL)
Assignee: SAMSUNG ELECTRONICS CO., LTD.
H04N5/343H04N5/347
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,570,384
App. No.
17/011,227
Granted
Jan 31, 2023
Kind
B2
Abstract

A method performed with an image sensor having a pixel array. At least one frame of a scene may be obtained using the pixel array. At least one region of interest (ROI) is identified within the frame. Subsequent frames of the scene are obtained, which involves controlling the pixel array to perform high resolution imaging with respect to the at least one ROI and low resolution imaging using analog binning with respect to remaining regions of the frames.

Claims (53)

1. A method performed with an image sensor having a pixel array, comprising:

obtaining at least one frame of a scene using the pixel array;

identifying at least one region of interest (ROI) within the at least one frame;

obtaining subsequent frames of the scene, which comprises controlling the pixel array to perform high resolution imaging with respect to the at least one ROI and low resolution imaging using analog binning with respect to remaining regions of the frames outside the at least one ROI;

providing image data obtained from the pixel array in pipelines according to the same resolution;

digitally processing each of the pipelines separately, which includes defect correction processing, noise reduction processing, and/or remosaicing processing, to provide a low resolution group of image data and at least one high resolution group of image data; and

stitching together the low resolution and high resolution groups of image data to form a unified frame to be displayed.

2. The method of claim 1 , further comprising:

forming a picture in picture (PIP) frame using the remaining regions represented in the unified frame as background image elements and at least one high resolution window corresponding to the at least one high resolution group of image data as a foreground image.

3. The method of claim 2 , further comprising enlarging the at least one high resolution window relative to its original size in relation to the background image elements.

4. The method of claim 1 , wherein the identifying of at least one ROI is performed automatically through commands initiated by a processor executing program instructions to identify particular patterns of image elements as ROIs.

5. The method of claim 1 , wherein the identifying at least one ROI is performed through a command initiated by a user interface.

6. The method of claim 1 , wherein:

the at least one ROI comprises a first ROI and a second ROI;

a first analog binning factor is used for imaging by the pixel array with respect to the first ROI; and

a second, lower binning factor is used for imaging by the pixel array with respect to the second ROI.

7. The method of claim 6 , wherein the first ROI comprises a vehicle and the second ROI comprises a license plate of the vehicle.

8. The method of claim 6 , wherein the first ROI comprises a human body and the second ROI comprises a face of the human body.

9. The method of claim 1 , wherein the high resolution imaging with respect to the at least one ROI is performed without analog binning.

10. The method of claim 1 , wherein no analog binning is applied with respect to the at least one ROI and analog binning factors of two or higher are applied with respect to the remaining regions of the frames.

11. The method of claim 1 , wherein:

the at least one ROI comprises a first ROI and a second ROI separated from the first ROI by a background region; and

an identical high resolution binning factor is used for imaging by the pixel array with respect to each of the first and second ROIs.

12. The method of claim 1 , wherein said digitally processing each of the pipelines separately comprises the defect correction processing.

13. The method of claim 1 , wherein the analog binning comprises:

defining a pixel block of adjacent pixels; and

at least one of (i) summing; (ii) averaging; and (iii) weighting charges from the pixels of the pixel block; and/or (iv) skipping pixel data of some of the adjacent pixels of the pixel block.

14. The method of claim 13 , wherein the digital binning is performed in an analogous manner as the analog binning using the at least one of the (i) summing; (ii) averaging; (ii) weighting; and (iv) skipping.

15. The method of claim 1 , wherein said digitally processing each of the pipelines separately includes the remosaicing processing.

16. The method of claim 1 , wherein said digitally processing each of the pipelines separately includes the noise reduction processing.

17. A method performed with an image sensor having a pixel array, comprising:

obtaining at least one frame of a scene using the pixel array;

identifying at least one region of interest (ROI) within the at least one frame;

obtaining subsequent frames of the scene, which comprises controlling the pixel array to perform high resolution imaging with respect to the at least one ROI and low resolution imaging using analog binning with respect to remaining regions of the frames outside the at least one ROI;

providing image data obtained from the pixel array in pipelines according to the same resolution;

digitally processing each of the pipelines separately to provide a low resolution group of image data and at least one high resolution group of image data;

stitching together the low resolution and high resolution groups of imaging data to form a unified frame to be displayed; and

prior to stitching together the low resolution and high resolution groups of image data, digitally binning the at least one high resolution group of image data at a binning factor sufficient for matching the low resolution imaging based on the analog binning;

wherein the unified frame is stitched together to form a display frame at the same resolution throughout.

18. An image sensor comprising:

a pixel array; and

at least one image processor executing instructions to:

obtain at least one frame of image data of a scene read out from the pixel array;

obtain subsequent frames of the scene, which comprises controlling the pixel array to perform high resolution imaging with respect to at least one region of interest (ROI) of the scene, and low resolution imaging using analog binning with respect to remaining regions of the frames outside the at least one ROI; and

digitally bin the at least one high resolution group of image data at a binning factor sufficient for matching the low resolution imaging based on the analog binning, to provide digitally binned image data.

19. The image sensor of claim 18 , further comprising:

providing image data obtained from the pixel array in pipelines according to the same resolution; and

digitally process each of the pipelines separately to provide a low resolution group of image data and at least one high resolution group of image data, and

wherein the at least one image processor further executes instructions to identify and track the at least one ROI.

20. The image sensor of claim 18 , wherein the at least one image processor further executes instructions to receive boundary information of the at least one ROI from a processor external to the image sensor.

21. The image sensor of claim 18 , wherein the at least one image processor further executes instructions to:

stitch together the low resolution group of image data and the digitally binned image data to form a unified frame to be displayed; and

generate a picture in picture (PIP) frame using the remaining regions represented in the unified frame as background image elements and at least one high resolution window corresponding to the at least one high resolution group of image data as a foreground image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 3, 2020
From: DUENYAS, SHAHAF; YAFFE, YOEL; HOROWITZ, GUY; EISENBERG, AMIT; HAMAMI, SHY; MONZON, ODED; BITAN, GAL; PIEPSH, YOAV
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 053685/0332 →
Continuity (1)
Related Publication 20220070391A1 · Mar 3, 2022