Neural network based auto-white-balancing
A method of auto white balancing, including, receiving an original image, determining an RG logarithmic ratio of a set of red to green channel values of the original image, determining a BG logarithmic ratio of a set of blue to green channel values of the original image, determine an original two-dimensional histogram utilizing the RG logarithmic ratio and the BG logarithmic ratio, determine a Gaussian-blur two-dimensional histogram utilizing the RG logarithmic ratio and the BG logarithmic ratio, determining a sharpened two-dimensional histogram of a sharpened image utilizing the RG logarithmic ratio and the BG logarithmic ratio, determining a Laplacian-edge two-dimensional histogram of a Laplacian-edge image utilizing the RG logarithmic ratio and the BG logarithmic ratio and determining a white balancing gain utilizing a neural network based on the original 2D histogram, the Gaussian-blur 2D histogram, the sharpened 2D histogram and the Laplacian-edge 2D histogram.
1. A method of auto white balancing, comprising:
receiving an original image;
determining an RG logarithmic ratio of a set of red channel values (R) to a set of green channel values (G) of the original image;
determining a BG logarithmic ratio of a set of blue channel values (B) to the set of green channel values (G) of the original image;
determining an original two-dimensional (2D) histogram of the original image utilizing the RG logarithmic ratio and the BG logarithmic ratio;
determining a Gaussian-blur two-dimensional (2D) histogram of a Gaussian-blur image of the original image utilizing the RG logarithmic ratio and the BG logarithmic ratio;
determining a sharpened two-dimensional (2D) histogram of a sharpened image of the original image utilizing the RG logarithmic ratio and the BG logarithmic ratio;
determining a Laplacian-edge two-dimensional (2D) histogram of a Laplacian-edge image of the original image utilizing the RG logarithmic ratio and the BG logarithmic ratio; and
determining a white balancing gain utilizing a neural network based on the original 2D histogram, the Gaussian-blur 2D histogram, the sharpened 2D histogram and the Laplacian-edge 2D histogram.
2. The method of auto white balancing of claim 1 , wherein the original image is a linear raw image.
3. The method of auto white balancing of claim 1 , further comprising training the neural network using a ground truth patch array.
4. The method of auto white balancing of claim 3 , further comprising calibrating a new sensor to the original image utilizing a transformation matrix based on a linear regression of the new sensor to the original image.
5. The method of auto white balancing of claim 1 , further comprising training the neural network using a mobile device camera.
6. The method of auto white balancing of claim 1 , wherein the original 2D histogram, the Gaussian-blur 2D histogram, the sharpened 2D histogram and the Laplacian-edge 2D histogram have 256 bins in both dimensions.
7. The method of auto white balancing of claim 1 , wherein a neural network last layer is a fully connected 256 by 256 and outputs the white balancing gain.
8. The method of auto white balancing of claim 1 , wherein a neural network activation function is ReLU.
9. The method of auto white balancing of claim 1 , wherein a neural network fully connected layer has leaky ReLU having an alpha of 0.125.
10. The method of auto white balancing of claim 1 , wherein the neural network is a U-Net.