White balance with reference illuminants
Introduced here are computer programs and associated computer-implemented techniques for achieving high-fidelity color reproduction in the absence of any known reflectance spectrums. That is, high-fidelity color reproduction can be achieved without portable references, such as gray cards and color checkers. To accomplish this, a new reference spectrum—the “reference illuminant spectrum”—is introduced into scenes to be imaged by image sensors. The reference illuminant spectrum is created by a multi-channel light source whose spectral properties are known.
1 . A method comprising:
receiving input that is indicative of a request to image a scene using a Red-Green-Blue (RGB) image sensor of an electronic device;
causing
(a) a first image of a scene to be captured by the RGB image sensor, with the electronic device performing an automatic white balance on the first image,
(b) a second image of the scene to be captured by the RGB image sensor, with the electronic device performing a fixed white balance on the second image, and
(c) a first series of images of the scene to be captured by the RGB image sensor while the scene is differentially illuminated, with each image in the first series of images corresponding to a different illuminant spectrum;
creating a second series of images by subtracting red, green, and blue values of each pixel in the second image from red, green, and blue values of a corresponding pixel from each image in the first series of images;
converting the second series of images into a CIELAB color space so that each pixel is represented by a series of a* values and a series of b* values;
converting the first image into the CIELAB color space so that each pixel is represented as an a* value and a b* value;
forming, for each of multiple pixels in the first image,
(i) a first linear equation based on a corresponding a* value from the converted first image and a corresponding series of a* values from the converted second series of images, and
(ii) a second linear equation based on a corresponding b* value from the converted first image and a corresponding series of b* values from the converted second series of images;
performing, for each of the multiple pixels in the first image, a least squares optimization on the first and second linear equations to produce a vector of coefficients, such that multiple vectors of coefficients are produced for the multiple pixels; and
populating the multiple vectors of coefficients into a data structure.
2 . The method of claim 1 , wherein the RGB image sensor is included in a front-facing camera of the electronic device.
3 . The method of claim 1 , further comprising:
addressing each color channel of a light source that has at least three color channels to perform a series of illumination events,
wherein each image in the first series of images is captured in conjunction with a different one of the series of illumination events.
4 . The method of claim 3 , wherein each color channel includes one or more illuminants that are configured to produce light of a substantially similar color.
5 . The method of claim 1 , wherein to perform the fixed white balance, the electronic device uses one of multiple preset white balances that are available to the electronic device.
6 . The method of claim 1 , wherein the first image is captured over a first exposure interval, and wherein the second image is captured over a second exposure interval that is less than the first exposure interval.
7 . The method of claim 1 , further comprising:
presenting an instruction to introduce, to the scene, an object with a known reflectance spectrum.
8 . A non-transitory medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:
receiving input that is indicative of a request to image a scene using a Red-Green-Blue (RGB) image sensor of an electronic device;
causing
(a) a first image of a scene to be captured by the RGB image sensor, with the electronic device performing an automatic white balance on the first image,
(b) a second image of the scene to be captured by the RGB image sensor, with the electronic device performing a fixed white balance on the second image, and
(c) a series of images of the scene to be captured by the RGB image sensor while the scene is differentially illuminated, with each image in the first series of images corresponding to a different illuminant spectrum; and
computing a matrix, with which the processor is able to alter red values, green values, and/or blue values for a given image to more accurately represent color, by—
creating a second series of images by subtracting the red, green, and blue values of each pixel in the second image from the red, green, and blue values of a corresponding pixel from each image in the series of images,
converting the second series of images into a CIELAB color space so that each pixel is represented by a series of a* values and a series of b* values,
converting the first image into the CIELAB color space so that each pixel is represented as an a* value and a b* value, forming, for each of multiple pixels in the first image,
(i) a first equation based on a corresponding a* value from the converted first image and a corresponding series of a* values from the converted second series of images, and
(ii) a second equation based on a corresponding b* value from the converted first image and a corresponding series of b* values from the converted second series of images,
optimizing, for each of the multiple pixels in the first image, the first and second equations to produce a vector of coefficients, such that multiple vectors of coefficients are produced for the multiple pixels, and
populating the multiple vectors of coefficients into the matrix.
9 . The non-transitory medium of claim 8 , wherein the operations further comprise:
populating, for each of the multiple pixels in the first image, a second vector that is representative of a chromaticity fingerprint with the corresponding series of a* values and the corresponding series of b* values.
10 . The non-transitory medium of claim 8 , wherein each illuminant spectrum is representative of light in multiple colors with one color at a higher intensity than the other colors.