IP Library › Granted Patent US 11,025,842
Granted Patent B2
US 11,025,842 · App. 16/376,426 · Granted Jun 1, 2021

Binner circuit for image signal processor

Inventors: Sheng Lin (San Jose, CA); D. Amnon Silverstein (Cupertino, CA)
Assignee: Apple Inc.
H04N5/347G06T5/002H04N5/23212H04N5/3456H04N5/367H04N9/0451H04N9/646
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,025,842
App. No.
16/376,426
Granted
Jun 1, 2021
Kind
B2
Abstract

Embodiments relate to image signal processors (ISP) that include binner circuits that down-sample an input image. An input image may include a plurality of pixels. The output image of the binner circuit may include a reduced number of pixels. The binner circuit may include a plurality of different operation modes. In a bin mode, the binner circuit may blend a subset of input pixel values to generate an output pixel quad. In a skip mode, the binner circuit may select one of the input pixel values as the output pixel pixel. The selection may be performed randomly to avoid aliasing. In a luminance mode, the binner circuit may take a weighted average of a subset of pixel values having different colors. In a color value mode, the binner circuit may select one of the colors in a subset of pixel values as an output pixel value.

Claims (60)

1. An image signal processor, comprising:

a binner circuit configured to:

receive raw image data comprising a plurality of input pixels;

select, for each output pixel of an output image of the binner circuit, a plurality of input pixel quads from the raw image data;

in a bin mode of the binner circuit and for each output pixel of the output image, blend the plurality of input pixel quads to generate a blended output pixel quad;

in a skip mode of the binner circuit and for each output pixel of the output image, select one of the plurality of input pixel quads to generate a selected output pixel quad;

in a luminance mode of the bin mode or the skip mode:

determine, for each output pixel, a weighted average of values of the blended output pixel quad or the selected output pixel quad, the values comprising three different colors in the blended output pixel quad or the selected output pixel quad, the weighted average being the output pixel of the output image; and

a statistics circuit operatively coupled to the binner circuit and configured to generate statistics on a version of the output image of the binner circuit.

2. The image signal processor of claim 1 , wherein the binner circuit is further configured to receive a mode instruction to place the binner circuit in the bin mode or the skip mode.

3. The image signal processor of claim 1 , wherein the binner circuit is further configured to:

in a color value mode of the bin mode or the skip mode:

receive a selection of a location, and

select, according to the selection of the location and for each output pixel, a pixel value from the blended output pixel quad or the selected output pixel quad, the pixel value being the output pixel of the output image.

4. The image signal processor of claim 1 , wherein, in the skip mode and for each pixel of the output image, the binner circuit is further configured to:

determine a sum of least significant bits of values of input pixels in the plurality of input pixel quads;

select, in response to the sum being an odd number, a first input pixel quad as the selected output pixel quad; and

select, in response to the sum being an even number, a second input pixel quad as the selected output pixel quad.

5. The image signal processor of claim 1 , wherein selection of the one of the plurality of input pixel quads in the skip mode is performed randomly.

6. The image signal processor of claim 1 , wherein the statistics correspond to an exposure level of an image sensor that produces the raw image data.

7. The image signal processor of claim 1 , wherein the binner circuit is configured to generate a single value for each output pixel of the output image, each of the single values corresponding to the plurality of input pixel quads in the raw image data.

8. A method of operating an image signal processor comprising a binner circuit, the method comprising:

receiving raw image data comprising a plurality of input pixels;

selecting, for each output pixel of an output image of the binner circuit, a plurality of input pixel quads from the raw image data;

blending, in a bin mode of the binner circuit and for each output pixel of the output image, the plurality of input pixel quads to generate a blended output pixel quad;

selecting, in a skip mode of the binner circuit and for each output pixel of the output image, one of the plurality of input pixel quads to generate a selected output pixel quad;

determining, in a luminance mode of the bin mode or the skip mode and for each output pixel, a weighted average of values of the blended output pixel quad or the selected output pixel quad, the values comprising three different colors in the blended output pixel quad or the selected output pixel quad, the weighted average being the output pixel of the output image; and

generating statistics of the image sensor on a version of the output image of the binner circuit.

9. The method of claim 8 , further comprising:

receiving a mode instruction to place the binner circuit in the bin mode or the skip mode.

10. The method of claim 8 , further comprising:

in a color value mode of the bin mode or the skip mode:

receiving a selection of a location, and

selecting, according to the selection of the location and for each output pixel, a pixel value from the blended output pixel quad or the selected output pixel quad at the location selected, the pixel value being the output pixel of the output image.

11. The method of claim 8 , further comprising:

in the skip mode and for each pixel of the output image:

determining a sum of least significant bits of values of input pixels in the plurality of input pixel quads;

selecting, in response to the sum being an odd number, a first input pixel quad as the selected output pixel quad; and

selecting, in response to the sum being an even number, a second input pixel quad as the selected output pixel quad.

12. The method of claim 8 , wherein selecting one of the plurality of input pixel quads in the skip mode is performed randomly.

13. The method of claim 8 , further comprising:

controlling an exposure level of an image sensor that produces the raw image data responsive to the statistics.

14. The method of claim 8 , wherein a single value is generated for each output pixel of the output image, each of the single values corresponding to the plurality of input pixel quads in the raw image data.

15. A system, comprising:

an image sensor; and

an image signal processor, comprising a binner circuit that is configured to:

receive raw image data from the image sensor, the raw image data comprising a plurality of input pixels;

select, for each output pixel of an output image of the binner circuit, a plurality of input pixel quads from the raw image data;

in a bin mode of the binner circuit and for each output pixel of the output image, blend the plurality of input pixel quads to generate a blended output pixel quad, and

in a skip mode of the binner circuit and for each output pixel of the output image, select one of the plurality of input pixel quads to generate a selected output pixel quad; and

in a luminance mode of the bin mode or the skip mode:

determine, for each output pixel, a weighted average of values of the blended output pixel quad or the selected output pixel quad, the values comprising three different colors in the blended output pixel quad or the selected output pixel quad, the weighted average being the output pixel of the output image.

16. The system of claim 15 , wherein the image signal processor further comprises a statistics circuit operatively coupled to the binner circuit and configured to generate statistics on a version of the output image of the binner circuit, the statistics corresponding to an exposure level of the image sensor.

17. The system of claim 15 , wherein the binner circuit is further configured to:

in a color value mode of the bin mode or the skip mode:

receive a selection of a location, and

select, according to the selection of the location and for each output pixel, a pixel value from the blended output pixel quad or the selected output pixel quad, the pixel value being the output pixel of the output image.

18. The image signal processor of claim 1 , wherein the luminance mode is a sub-mode of the bin mode or the skip mode or the luminance mode is downstream of the bin mode or the skip mode.

19. The method of claim 8 , wherein the luminance mode is a sub-mode of the bin mode or the skip mode or the luminance mode is downstream of the bin mode or the skip mode.

20. The system of claim 15 , wherein the luminance mode is a sub-mode of the bin mode or the skip mode or the luminance mode is downstream of the bin mode or the skip mode.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 8, 2019
From: LI, SHENG; SILVERSTEIN, D.AMNON
To: APPLE INC.
Reel/Frame 048819/0394 →
Continuity (1)
Related Publication 20200322549A1 · Oct 8, 2020
Cited By (1)
US 12,356,103