On-camera image processing based on image luminance data
A camera system processes images based on image luminance data. The camera system includes an image sensor, an image pipeline, an encoder and a memory. The image sensor converts light incident upon the image sensor into raw image data. The image pipeline converts raw image data into color-space image data and calculates luminance levels of the color-space image data. The encoder can determine one or more of quantization levels, determining GOP structure or reference frame spacing for the color-space image data based on the luminance levels. The memory stores the color-space image data and the luminance levels.
1. A camera system, comprising:
an image sensor configured to convert light incident upon the image sensor into raw image data;
a processing apparatus that controls operation and functionality for the camera system; and
a non-transitory computer readable apparatus comprising a storage medium having a plurality of instructions stored thereon, the plurality of instructions being configured to, when executed by the processing apparatus, cause production of encoded image data by:
conversion of the raw image data into color-space image data;
calculation of luminance levels of the color-space image data;
storage of the color-space image data and the calculated luminance levels of the color-space image data;
retrieval of the stored color-space image data and the calculated luminance levels;
determination of quantization levels of the color-space image data based on the calculated luminance levels;
an encode of the color-space image data using the determined quantization levels to produce the encoded image data; and
storage of the encoded image data.
2. The camera system of claim 1 , wherein the non-transitory computer readable apparatus comprises a combination of software and hardware.
3. The camera system of claim 2 , wherein the calculation of the luminance levels of the color-space image data comprises:
an identification of a plurality of blocks of the color-space image data; and
a calculation of a luminance level for each block of the plurality of blocks.
4. The camera system of claim 3 , wherein the calculation of the luminance level for each block comprises a calculation of a sum of pixel values for each block.
5. The camera system of claim 3 , wherein the determination of the quantization levels of the color-space image data based on the calculated luminance levels comprises:
a determination of a quantization level for each block based on a corresponding luminance level of the block.
6. The camera system of claim 5 , wherein a greater quantization level is determined for a first block than for a second block that is darker than the first block.
7. The camera system of claim 2 , wherein the non-transitory computer readable apparatus is further configured to cause production of encoded image data by calculation of activity variances of the color-space image data, determination of quantization levels of the color-space image data based on the activity variances, an encode of the color-space image data using the determined quantization levels, and storage of the color-space image data and the activity variances.
8. The camera system of claim 7 , wherein at least one quantization level is determined based on both an activity variance and a luminance level.
9. The camera system of claim 2 , wherein the non-transitory computer readable apparatus is further configured to cause production of encoded image data by determination of a reference frame and a group of pictures (GOP) structure based on the luminance levels.
10. The camera system of claim 2 , wherein the non-transitory computer readable apparatus is further configured to cause production of encoded image data by a pre-process of the encoded image data using one or more pre-processing operations.
11. A non-transitory computer readable apparatus comprising a storage medium having a plurality of instructions stored thereon, the plurality of instructions being configured to, when executed by a processing apparatus, cause production of encoded image data by:
a retrieval of raw image data;
a conversion of the raw image data into color-space image data;
a calculation of luminance levels of the color-space image data;
a determination of quantization levels of the color-space image data based on the calculated luminance levels; and
an encode of the color-space image data using the determined quantization levels to produce the encoded image data.
12. The non-transitory computer readable apparatus of claim 11 , wherein the calculation of the luminance levels of the color-space image data comprises:
an identification of a plurality of blocks of the color-space image data; and
a calculation of a luminance level for each block of the plurality of blocks.
13. The non-transitory computer readable apparatus of claim 12 , wherein the calculation of the luminance level of each block comprises:
a calculation of a sum of pixel values in each block.
14. The non-transitory computer readable apparatus of claim 12 , wherein the determination of the quantization levels of the color-space image data based on the luminance levels comprises:
a determination of a quantization level for each block based on a corresponding luminance level of the block.
15. The non-transitory computer readable apparatus of claim 14 , wherein a greater quantization level is determined for a first block than for a second block that is darker than the first block.
16. The non-transitory computer readable apparatus of claim 11 , wherein the plurality of instructions are further configured to, when executed by the processing apparatus, cause production of encoded image data by:
a calculation of activity variances of the color-space image data;
a determination of quantization levels of the color-space image data based on the calculated activity variances;
an encode of the color-space image data using the determined quantization levels; and
a storage of the color-space image data and the activity variances.
17. The non-transitory computer readable apparatus of claim 16 , wherein at least one quantization level is determined based on both an activity variance and a luminance level.
18. The non-transitory computer readable apparatus of claim 11 , wherein the plurality of instructions are further configured to, when executed by the processing apparatus, cause production of encoded image data by:
a storage of the color-space image data and the calculated luminance levels into a memory, wherein the color-space image data and the luminance levels are extracted from the memory before the quantization levels are determined.
19. The non-transitory computer readable apparatus of claim 11 , wherein the plurality of instructions are further configured to, when executed by the processing apparatus, cause production of encoded image data by a determination of a reference frame and a group of pictures (GOP) structure based on the calculated luminance levels.
20. The non-transitory computer readable apparatus of claim 11 , wherein the plurality of instructions are further configured to, when executed by the processing apparatus, cause production of encoded image data by a pre-process of the encoded image data using one or more pre-processing operations.