Dynamic global tone mapping with integrated 3D color look-up table
The processing of RGB image data can be optimized by performing optimization operations on the image data when it is converted into the YCbCr color space. For each Y-layer of the YCbCr image data, a 2D LUT is generated. The YCbCr image data is converted into optimized CbCr image data using the 2D LUTs, and optimized YCbCr image data is generated by blending CbCr image data corresponding to multiple Y-layers. The optimized YCbCr image data is converted into sRGB image data, and a tone curve is applied to the sRGB image data to produce optimized sRGB image data.
1. A camera system, comprising:
an image signal processor (“ISP”) configured to:
receive a YCbCr image data;
generate, for each of a plurality of Y-layers of the YCbCr image data, a 2D color look-up table (“LUT”);
convert the YCbCr image data into optimized CbCr image data using the 2D LUTs;
generate optimized YCbCr image data by blending, for at least two Y-layers, corresponding CbCr image data;
convert the optimized YCbCr image data into tone-mapped optimized sRGB image data; and
store the tone-mapped optimized sRGB image data in a memory of the camera system.
2. The camera system of claim 1 , wherein the ISP is further configured to:
convert a raw RGB color space into a YCbCr reference color space; and
convert raw RGB image data into the YCbCr image data using the YCbCr reference color space by transforming the raw RGB color space into an enhanced RGB color space using a linear transformation.
3. The camera system of claim 2 , wherein converting the raw RGB color
space into a YCbCr reference color space further comprises:
converting the enhanced RGB color space into a non-linear RGB color space using a non-linear conversion model.
4. The camera system of claim 3 , wherein converting the raw RGB color
space into a YCbCr reference color space further comprises:
transforming the non-linear RGB color space into the YCbCr reference color space in a xvYCC color space using a transformation matrix.
5. The camera system of claim 1 , wherein generating a 2D color LUT comprises:
accessing a set of reference grid points; and
estimating a transformation matrix for each reference grid point for different Y-layers.
6. The camera system of claim 5 , wherein converting the YCbCr image data into optimized CbCr image data comprises:
determining optimized CbCr points for each reference grid point for the different Y-layers using the estimated transformation matrices; and
transforming the YCbCr image data into optimized CbCr image data using the optimized CbCr points.
7. The camera system of claim 6 , wherein transforming the YCbCr image data into optimized CbCr image data using the optimized CbCr points comprises performing bilinear interpolation on the optimized CbCr points for different Y-layers.
8. The camera system of claim 1 , wherein generating a 2D color LUT comprises:
accessing a target YCbCr color space;
determining an optimized chroma gain map for different Y-layers based on a YCbCr reference color space and the target YCbCr color space; and
determining an optimized hue shift map for different Y-layers.
9. The camera system of claim 8 , wherein converting the YCbCr image data into optimized CbCr image data comprises:
converting the YCbCr image data into optimized CbCr image data using the optimized chroma gain maps and the optimized hue shift maps.
10. The camera system of claim 1 , wherein converting the optimized YCbCr image data into tone-mapped optimized sRGB image data comprises:
converting the optimized YCbCr image data into sRGB image data; and
applying a tone curve to the sRGB image data to produce tone-mapped optimized sRGB image data.
11. A method, comprising:
receiving, by an image signal processor (“ISP”), YCbCr image data;
generating, by the ISP, for each of a plurality of Y-layers of the YCbCr image data, a 2D color look-up table (“LUT”);
converting, by the ISP, the YCbCr image data into optimized CbCr image data using the 2D LUTs;
generating, by the ISP, optimized YCbCr image data by blending, for at least two Y-layers, corresponding CbCr image data;
converting, by the ISP, the optimized YCbCr image data into tone-mapped optimized sRGB image data; and
storing, by a memory, the tone-mapped optimized sRGB image data.
12. The method of claim 11 , further comprising:
converting, by a camera, light incident upon an image sensor of the camera into raw RGB image data;
accessing, by the ISP, a raw RGB color space corresponding to the raw RGB image data; and
converting, by the ISP, the raw RGB color space into a YCbCr reference color space by transforming the raw RGB color space into an enhanced RGB color space using a linear transformation.
13. The method of claim 12 , wherein converting the raw RGB color space into a YCbCr reference color space further comprises:
converting the enhanced RGB color space into a non-linear RGB color space using a non-linear conversion model.
14. The method of claim 13 , wherein converting the raw RGB color space into a YCbCr reference color space further comprises:
transforming the non-linear RGB color space into the YCbCr reference color space in a xvYCC color space using a transformation matrix.
15. The method of claim 11 , wherein generating a 2D color LUT comprises:
accessing a set of reference grid points; and
estimating a transformation matrix for each reference grid point for different Y-layers.
16. The method of claim 15 , wherein converting the YCbCr image data into optimized CbCr image data comprises:
determining optimized CbCr points for each reference grid point for the different Y-layers using the estimated transformation matrices; and
transforming the YCbCr image data into optimized CbCr image data using the optimized CbCr points.
17. The method of claim 16 , wherein transforming the YCbCr image data into optimized CbCr image data using the optimized CbCr points comprises performing bilinear interpolation on the optimized CbCr points for different Y-layers.
18. The method of claim 11 , wherein generating a 2D color LUT comprises:
accessing a target YCbCr color space;
determining an optimized chroma gain map for different Y-layers based on a YCbCr reference color space and the target YCbCr color space; and
determining an optimized hue shift map for different Y-layers.
19. The method of claim 18 , wherein converting the YCbCr image data into optimized CbCr image data comprises:
converting the YCbCr image data into optimized CbCr image data using the optimized chroma gain maps and the optimized hue shift maps.
20. The method of claim 11 , wherein converting the optimized YCbCr image data into tone-mapped optimized sRGB image data comprises:
converting the optimized YCbCr image data into sRGB image data; and
applying a tone curve to the sRGB image data to produce tone-mapped optimized sRGB image data.