On-camera image processing based on image luminance data
A camera system processes images based on image luminance data. The camera system includes an image sensor, an image pipeline, an encoder and a memory. The image sensor converts light incident upon the image sensor into raw image data. The image pipeline converts raw image data into color-space image data and calculates luminance levels of the color-space image data. The encoder can determine one or more of quantization levels, determining GOP structure or reference frame spacing for the color-space image data based on the luminance levels. The memory stores the color-space image data and the luminance levels.
1. A camera system, comprising:
an image sensor configured to convert light incident upon the image sensor into raw image data;
an image pipeline configured to:
convert the raw image data into color-space image data; and
calculate luminance levels of the color-space image data;
an encoder configured to:
determine quantization levels of the color-space image data based on the luminance levels; and
encode the color-space image data using the determined quantization levels to produce encoded image data; and
a memory configured to store the encoded image data.
2. The camera system of claim 1 , wherein calculating luminance levels of the color-space image data comprises:
identifying a plurality of blocks of the color-space image data; and
calculating a luminance level of each block.
3. The camera system of claim 2 , wherein calculating the luminance level of each block comprises:
calculating a sum of pixel values in each block.
4. The camera system of claim 2 , wherein determining quantization levels of the color-space image data based on the luminance levels comprises:
determining a quantization level for each block based on a corresponding luminance level of the block.
5. The camera system of claim 4 , wherein a greater quantization level is determined for a first block than for a second block darker than the first block.
6. The camera system of claim 1 , wherein the image pipeline is further configured to calculate activity variances of the color-space image data, wherein the encoder is further configured to determine quantization levels of the color-space image data based on the activity variances and to encode the color-space image data using the determined quantization levels, and wherein the memory further is configured to store the color-space image data and the activity variances.
7. The camera system of claim 6 , wherein at least one quantization level is determined based on both an activity variance and a luminance level.
8. The camera system of claim 1 , wherein the image pipeline is further configured to store the color-space image data and luminance levels into the memory, and the encoder is further configured to extract the color-space image data and luminance levels from the memory.
9. The camera system of claim 1 , wherein the encoder is further configured to determine a reference frame and GOP structure based on the luminance levels.
10. The camera system of claim 1 , further comprising a pre-encoding module configured to pre-process the encoded image data using one or more pre-processing operations.
11. A method, comprising:
capturing, by a camera, light incident upon an image sensor to produce raw image data;
converting, by the camera, the raw image data into color-space image data;
calculating, by the camera, luminance levels of the color-space image data;
determining, by the camera, quantization levels of the color-space image data based on the luminance levels; and
encoding, by the camera, the color-space image data using the determined quantization levels to produce encoded image data.
12. The method of claim 11 , wherein calculating luminance levels of the color-space image data comprises:
identifying a plurality of blocks of the color-space image data; and
calculating a luminance level of each block.
13. The method of claim 12 , wherein calculating the luminance level of each block comprises:
calculating a sum of pixel values in each block.
14. The method of claim 12 , wherein determining quantization levels of the color-space image data based on the luminance levels comprises:
determining a quantization level for each block based on a corresponding luminance level of the block.
15. The method of claim 14 , wherein a greater quantization level is determined for a first block than for a second block darker than the first block.
16. The method of claim 11 , further comprising:
calculating activity variances of the color-space image data;
determining quantization levels of the color-space image data further based on the activity variances;
encoding the color-space image data using the determined quantization levels; and
storing the color-space image data and the activity variances.
17. The method of claim 16 , wherein at least one quantization level is determined based on both an activity variance and a luminance level.
18. The method of claim 11 , further comprising storing the color-space image data and luminance levels into a memory, wherein the color-space image data and luminance levels are extracted from the memory before the quantization levels are determined.
19. The method of claim 11 , further comprising determining a reference frame and GOP structure based on the luminance levels.
20. The method of claim 11 , further comprising pre-processing the encoded image data using one or more pre-processing operations.