Color correction of RGBIR sensor stream based on resolution recovery of RGB and IR channels
Techniques are provided for color correction of image frames, received from a hybrid RGBIR sensor, using resolution recovery of the red, green, blue (RGB), and infra-red (IR) channels. The color correction is to compensate for IR contamination of the RGB channels. A methodology implementing the techniques according to an embodiment includes a demosaic operation on the RGBIR image frame to generate full resolution red, green, blue, and IR frames. The demosaic operation includes filtering to extract Chroma and Luma components of the RGBIR image to reconstruct the full resolution images. The method also includes calculating an IR weighting factor for the pixels of each of the full resolution RGB frames, and correcting for IR contamination of those pixels by subtracting a scaled value of the corresponding pixels from the full resolution IR frame. The scaled value is based on the IR weighting factor associated with the pixel to be corrected.
1. A processor-implemented method for color correction of a red-green-blue-infrared (RBGIR) image frame, the method comprising:
performing, by a processor, a demosaic operation on the RGBIR image frame to generate a full resolution red frame, a full resolution green frame, a full resolution blue frame, and a full resolution IR frame, wherein each of the frames comprise a plurality of pixels;
calculating, by the processor, an IR weighting factor for one or more of the pixels of each of the full resolution red, green, and blue frames, wherein a value of each of the IR weighting factors is affected by a location of the corresponding pixel in its respective full resolution frame; and
correcting, by the processor, for IR contamination of the pixels of each of the full resolution red, green, and blue frames, by subtracting from each of the pixels to be corrected, a scaled value of a corresponding pixel from the full resolution IR frame, the scaled value based on the IR weighting factor associated with the pixel to be corrected.
2. The method of claim 1 , further comprising performing a remosaic operation on the corrected full resolution red, green, and blue frames to generate a color corrected Bayer image frame.
3. The method of claim 1 , wherein the demosaic operation further comprises:
applying high pass filters to the RGBIR image frame to extract three Chroma image components;
subtracting the three Chroma image components from the RGBIR image frame to obtain a Luma image component; and
generating pixels of the full resolution red, green, and blue frames as a weighted combination of the pixels of the three Chroma image components and the Luma image component.
4. The method of claim 3 , wherein the demosaic operation further comprises applying a noise reduction filter to the three Chroma image components, wherein the noise reduction filter averages neighboring pixels.
5. The method of claim 3 , wherein the demosaic operation further comprises applying horizontal and vertical directional interpolation filters to the Luma image component.
6. The method of claim 3 , wherein the demosaic operation further comprises applying a gradient filter to the Luma image component.
7. The method of claim 1 , wherein an RGBIR sensor generates the RGBIR image frame, and the IR weighting factors are based on a response of red, green, and blue channels of the RGBIR sensor to light in the IR spectrum.
8. The method of claim 1 , wherein the IR weighting factors are largest for those pixels located near the corners of the corresponding full resolution frame.
9. A system for color correction of a red-green-blue-infrared (RBGIR) image frame, the system comprising:
a demosaic circuit configured to perform a demosaic operation on the RGBIR image frame to generate a full resolution red frame, a full resolution green frame, a full resolution blue frame, and a full resolution IR frame, wherein each of the frames comprise a plurality of pixels;
an IR weighting circuit configured to calculate an IR weighting factor for one or more of the pixels of each of the full resolution red, green, and blue frames, wherein a value of each of the IR weighting factors is affected by a location of the corresponding pixel in its respective full resolution frame; and
an IR compensation circuit configured to correct for IR contamination of the pixels of each of the full resolution red, green, and blue frames, by subtracting from each of the pixels to be corrected, a scaled value of a corresponding pixel from the full resolution IR frame, the scaled value based on the IR weighting factor associated with the pixel to be corrected.
10. The system of claim 9 , further comprising a remosaic circuit configured to perform a remosaic operation on the corrected full resolution red, green, and blue frames to generate a color corrected Bayer image frame.
11. The system of claim 9 , wherein the demosaic circuit further comprises:
a Chroma filter circuit configured to high pass filter the RGBIR image frame to extract three Chroma image components;
a Luma calculation circuit configured to subtract the three Chroma image components from the RGBIR image frame to obtain a Luma image component; and
full resolution calculation circuit configured to generate pixels of the full resolution red, green, and blue frames as a weighted combination of the pixels of the three Chroma image components and the Luma image component.
12. The system of claim 11 , wherein the demosaic circuit further comprises a noise reduction filter circuit configured to average the three Chroma image components of neighboring pixels.
13. The system of claim 11 , wherein the demosaic circuit further comprises a directional filter circuit configured to apply horizontal and vertical directional interpolation filters to the Luma image component.
14. The system of claim 11 , wherein the demosaic circuit further comprises a gradient blending circuit configured to apply a gradient filter to the Luma image component.
15. The system of claim 9 , wherein an RGBIR sensor generates the RGBIR image frame, and the IR weighting factors are based on a response of red, green, and blue channels of the RGBIR sensor to light in the IR spectrum.
16. The system of claim 9 , wherein the IR weighting factors are largest for those pixels located near the corners of the corresponding full resolution frame.
17. At least one non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, result in the following operations for color correction of a red-green-blue-infrared (RBGIR) image frame, the operations comprising:
performing a demosaic operation on the RGBIR image frame to generate a full resolution red frame, a full resolution green frame, a full resolution blue frame, and a full resolution IR frame, wherein each of the frames comprise a plurality of pixels;
calculating an IR weighting factor for one or more of the pixels of each of the full resolution red, green, and blue frames, wherein a value of each of the IR weighting factors is affected by a location of the corresponding pixel in its respective full resolution frame; and
correcting for IR contamination of the pixels of each of the full resolution red, green, and blue frames, by subtracting from each of the pixels to be corrected, a scaled value of a corresponding pixel from the full resolution IR frame, the scaled value based on the IR weighting factor associated with the pixel to be corrected.
18. The computer readable storage medium of claim 17 , the operations further comprising performing a remosaic operation on the corrected full resolution red, green, and blue frames to generate a color corrected Bayer image frame.
19. The computer readable storage medium of claim 17 , wherein the demosaic operation further comprises:
applying high pass filters to the RGBIR image frame to extract three Chroma image components;
subtracting the three Chroma image components from the RGBIR image frame to obtain a Luma image component; and
generating pixels of the full resolution red, green, and blue frames as a weighted combination of the pixels of the three Chroma image components and the Luma image component.
20. The computer readable storage medium of claim 19 , wherein the demosaic operation further comprises applying a noise reduction filter to the three Chroma image components, wherein the noise reduction filter averages neighboring pixels.
21. The computer readable storage medium of claim 19 , wherein the demosaic operation further comprises applying horizontal and vertical directional interpolation filters to the Luma image component.
22. The computer readable storage medium of claim 19 , wherein the demosaic operation further comprises applying a gradient filter to the Luma image component.
23. The computer readable storage medium of claim 17 , wherein an RGBIR sensor generates the RGBIR image frame, and the IR weighting factors are based on a response of red, green, and blue channels of the RGBIR sensor to light in the IR spectrum.
24. The computer readable storage medium of claim 17 , wherein the IR weighting factors are largest for those pixels located near the corners of the corresponding full resolution frame.