Image demosaicing method

1 . A method for demosaicing an image captured by an image sensor having a pixel array that includes a first Bayer unit, the method comprising:

determining, for a red pixel of the first Bayer unit and with circuitry communicatively coupled to the pixel array, a first red image-pixel value, a first green image-pixel value, and a first blue image-pixel value such that:

a first weighted sum of the first red, the first green, and the first blue image-pixel values equals a red sensor-pixel value output by the red pixel;

a second weighted sum of the first red, the first green, and the first blue image-pixel values equals an average of a first green sensor-pixel value output by a first green pixel of the first Bayer unit and a second green sensor-pixel value output by a second green pixel of the first Bayer unit; and

a third weighted sum of the first red, the first green, and the first blue image-pixel values equals a blue sensor-pixel value output by a blue pixel of the first Bayer unit; and

outputting, by the circuitry, a demosaiced image having, at a location in the demosaiced image corresponding to a location of the red pixel in the pixel array, a first RGB triplet that includes the first red image-pixel value, the first green image-pixel value, and the first blue image-pixel value.

2 . The method of claim 1 , the image sensor including a color-filter array,

the red pixel being aligned beneath a red color filter of the color-filter array;

each of the first and the second green pixels being aligned beneath a green color filter of the color-filter array; and

the blue pixel being aligned beneath a blue color filter of the color-filter array;

wherein:

in the first weighted sum, the first red, the first green, and the first blue image-pixel values are weighted by respective weights w 11 , w 12 , and w 13 , which are derived from a transmission spectrum of the red color filter at a red wavelength, a green wavelength, and a blue wavelength, respectively;

in the second weighted sum, the first red, the first green, and the first blue image-pixel values are weighted by respective weights w 21 , w 22 , and w 23 , which are derived from a transmission spectrum of the green color filter at the red wavelength, the green wavelength, and the blue wavelength, respectively; and

in the third weighted sum, the first red, the first green, and the first blue image-pixel values are weighted by respective weights w 31 , w 32 , and w 33 , which are derived from a transmission spectrum of the blue color filter at the red wavelength, the green wavelength, and the blue wavelength, respectively.

3 . The method of claim 1 , further comprising:

reading, with readout circuitry of the image sensor, each of the red sensor-pixel value, the first green sensor-pixel value, the second green sensor-pixel value, and the blue sensor-pixel value from the pixel array.

4 . The method of claim 1 , the circuitry being an image signal processor that is part of the image sensor or communicatively coupled to the image sensor.

5 . The method of claim 1 ,

the red pixel being at an upper-left corner of the first Bayer unit;

the first green pixel being at an upper-right corner of the first Bayer unit;

the second green pixel being at a lower-left corner of the first Bayer unit; and

the blue pixel being at a lower-right corner of the first Bayer unit.

6 . The method of claim 5 , the pixel array including a second Bayer unit that partially overlaps the first Bayer unit and includes the first green pixel, the blue pixel, a second red pixel, and a third green pixel, the method further comprising:

determining, for the first green pixel of the first Bayer unit and with the circuitry, a second red image-pixel value, a second green image-pixel value, and a second blue image-pixel value such that:

a fourth weighted sum of the second red, the second green, and the second blue image-pixel values equals a second red sensor-pixel value output by a second red pixel;

a fifth weighted sum of the second red, the second green, and the second blue image-pixel values equals an average of the first green sensor-pixel value and a third green sensor-pixel value output by the third green pixel; and

a sixth weighted sum of the second red, the second green, and the second blue image-pixel values equals the blue sensor-pixel value;

wherein, in said outputting, the demosaiced image having, at a location in the demosaiced image corresponding to a location of the first green pixel in the pixel array, a second RGB triplet that includes the second red image-pixel value, the second green image-pixel value, and the second blue image-pixel value.

7 . The method of claim 6 ,

the first green pixel being at the upper-left corner of the second Bayer unit;

the second red pixel being at the upper-right corner of the second Bayer unit;

the blue pixel being at the lower-left corner of the second Bayer unit; and

the third green pixel being at the lower-right corner of the second Bayer unit.

8 . The method of claim 6 , the image sensor including a color-filter array,

the second red pixel being aligned beneath a red color filter of the color-filter array;

second green pixel being aligned beneath a green color filter of the color-filter array; and

wherein:

in the fourth weighted sum, the second red, the first green, and the first blue image-pixel values are weighted by respective weights w 11 , w 12 , and w 13 , which are derived from a transmission spectrum of the red color filter at a red wavelength, a green wavelength, and a blue wavelength, respectively;

in the fifth weighted sum, the second red, the first green, and the first blue image-pixel values are weighted by respective weights w 21 , w 22 , and w 23 , which are derived from a transmission spectrum of the green color filter at the red wavelength, the green wavelength, and the blue wavelength, respectively; and

in the sixth weighted sum, the second red, the first green, and the first blue image-pixel values are weighted by respective weights w 31 , w 32 , and w 33 , which are derived from a transmission spectrum of the blue color filter at the red wavelength, the green wavelength, and the blue wavelength, respectively.

9 . A method for demosaicing an image captured by an image sensor having a pixel array that includes a first Bayer unit, the method comprising:

determining, for a red pixel of the first Bayer unit and with circuitry communicatively coupled to the pixel array, a first red image-pixel value, a first green image-pixel value, a first blue image-pixel value, and first infrared (IR) image-pixel value such that:

a first weighted sum of the first red, the first green, the first blue, and the first IR image-pixel values equals a red sensor-pixel value output by the red pixel;

a second weighted sum of the first red, the first green, the first blue, and the first IR image-pixel values equals a green sensor-pixel value output by a green pixel of the first Bayer unit;

a third weighted sum of the first red, the first green, the first blue, and the first IR image-pixel values equals a blue sensor-pixel value output by a blue pixel of the first Bayer unit; and

a fourth weighted sum of the first red, the first green, the first blue, and the first IR image-pixel values equals an IR sensor-pixel value output by an IR pixel of the first Bayer unit; and

outputting, by the circuitry, a demosaiced image having, at a location in the demosaiced image corresponding to a location of the red pixel in the pixel array, a first RGB quadruplet that includes the first red image-pixel value, the first green image-pixel value, first blue image-pixel value, and the first IR image-pixel value.

10 . The method of claim 9 , the red pixel being at an upper-left corner of the first Bayer unit;

the green pixel being at an upper-right corner of the first Bayer unit;

the IR pixel being at a lower-left corner of the first Bayer unit; and

the blue pixel being at a lower-right corner of the first Bayer unit.

11 . The method of claim 10 , the pixel array including a second Bayer unit that partially overlaps the first Bayer unit and includes the green pixel, the blue pixel, a second red pixel, and a second IR pixel, the method further comprising:

determining, for the green pixel of the first Bayer unit and with the circuitry, a second red image-pixel value, a second green image-pixel value, a second blue image-pixel value, and a second IR image-pixel value such that:

a fifth weighted sum of the second red, the second green, the second blue, and the second IR image-pixel values equals the green sensor-pixel value;

a sixth weighted sum of the second red, the second green, the second blue, and the second IR image-pixel values equals a second red sensor-pixel value output by the second red pixel;

a seventh weighted sum of the second red, the second green, the second blue, and the second IR image-pixel values equals a second IR sensor-pixel value output by the second IR pixel; and

an eighth weighted sum of the second red, the second green, the second blue, and the second IR image-pixel values equals the blue sensor-pixel value;

wherein, in said outputting, the demosaiced image having, at a location in the demosaiced image corresponding to a location of the green pixel in the pixel array, a second RGB quadruplet that includes the second red image-pixel value, the second green image-pixel value, the second blue image-pixel value, and the second IR image-pixel value.

12 . The method of claim 11 ,

the green pixel being at an upper-left corner of the second Bayer unit;

the second red pixel being at an upper-right corner of the second Bayer unit;

the blue pixel being at a lower-left corner of the second Bayer unit; and

the second IR pixel being at a lower-right corner of the second Bayer unit.

13 . The method of claim 9 , the image sensor including a color-filter array,

the red pixel being aligned beneath a red color filter of the color-filter array;

the green pixel being aligned beneath a green color filter of the color-filter array;

the blue pixel being aligned beneath a blue color filter of the color-filter array; and

the IR pixel being aligned beneath an IR color filter of the color-filter array

wherein:

in the first weighted sum, the first red, the first green, the first blue, and the first IR image-pixel values are weighted by respective weights w 11 , w 12 , w 13 , and w 14 , which are derived from a transmission spectrum of the red color filter at a red wavelength, a green wavelength, a blue wavelength, and an IR wavelength, respectively;

in the second weighted sum, the first red, the first green, the first blue, and the first IR image-pixel values are weighted by respective weights w 21 , w 22 , w 23 , and w 24 , which are derived from a transmission spectrum of the green color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively;

in the third weighted sum, the first red, the first green, the first blue, and the first IR image-pixel values are weighted by respective weights w 31 , w 32 , w 33 , and w 34 , which are derived from a transmission spectrum of the blue color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively; and

in the fourth weighted sum, the first red, the first green, the first blue, and the first IR image-pixel values are weighted by respective weights w 41 , w 42 , and w 43 , and w 44 , which are derived from a transmission spectrum of the IR color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively.

14 . The method of claim 9 , further comprising:

reading, with readout circuitry of the image sensor, each of the red sensor-pixel value, the green sensor-pixel value, the blue sensor-pixel value, and the IR sensor-pixel value from the pixel array.

15 . The method of claim 9 , the circuitry being an image signal processor that is part of the image sensor or communicatively coupled to the image sensor.

16 . The method of claim 11 , the image sensor including a color-filter array, the green pixel being aligned beneath a green color filter of the color-filter array;

the blue pixel being aligned beneath a blue color filter of the color-filter array;

the second red pixel being aligned beneath a red color filter of the color-filter array; and

the second IR pixel being aligned beneath an IR color filter of the color-filter array

wherein:

in the fifth weighted sum, the second red, the second green, the second blue, and the second IR image-pixel values are weighted by respective weights w 11 , w 12 , w 13 , and w 14 , which are derived from a transmission spectrum of the red color filter at a red wavelength, a green wavelength, a blue wavelength, and an IR wavelength, respectively;

in the sixth weighted sum, the second red, the second green, the second blue, and the second IR image-pixel values are weighted by respective weights w 21 , w 22 , ad w 23 , and w 24 , which are derived from a transmission spectrum of the green color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively;

in the seventh weighted sum, the second red, the second green, the second blue, and the second IR image-pixel values are weighted by respective weights w 31 , w 32 , w 33 , and w 34 , which are derived from a transmission spectrum of the blue color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively; and

in the eighth weighted sum, the second red, the second green, the second blue, and the second IR image-pixel values are weighted by respective weights w 41 , w 42 , w 43 , and w 44 , which are derived from a transmission spectrum of the IR color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively.

17 . A method for demosaicing an image captured by an image sensor having a pixel array that includes a Bayer unit, the method comprising:

determining, for a red pixel of the Bayer unit and with circuitry communicatively coupled to the pixel array, a first red image-pixel value, a first green image-pixel value, and a first blue image-pixel value such that:

a first weighted sum of the first red, the first green, and the first blue image-pixel values equals a red sensor-pixel value output by the red pixel;

a second weighted sum of the first red, the first green, and the first blue image-pixel values equals a green sensor-pixel value output by a green pixel of the Bayer unit that is adjacent to the red pixel; and

a third weighted sum of the first red, the first green, and the first blue image-pixel values equals a blue sensor-pixel value output by a blue pixel of the Bayer unit that is adjacent to at least one of the red pixel and the blue pixel; and

outputting, by the circuitry, a demosaiced image having, at a location in the demosaiced image corresponding to a location of the red pixel in the pixel array, a first RGB triplet that includes the first red image-pixel value, the first green image-pixel value, and the first blue image-pixel value.

18 . The method of claim 17 , the image sensor including a color-filter array,

the red pixel, the green pixel, and the blue pixel being aligned, respectively, beneath a red color filter, a green color filter, and a blue color filter, of the color-filter array;

wherein:

in the first weighted sum, the first red, the first green, and the first blue image-pixel values are weighted by respective weights w 11 , w 12 , and w 13 , which are derived from a transmission spectrum of the red color filter at a red wavelength, a green wavelength, and a blue wavelength, respectively;

in the second weighted sum, the first red, the first green, and the first blue image-pixel values are weighted by respective weights w 21 , w 22 , and w 23 , which are derived from a transmission spectrum of the green color filter at the red wavelength, the green wavelength, and the blue wavelength, respectively; and

in the third weighted sum, the first red, the first green, and the first blue image-pixel values are weighted by respective weights w 31 , w 32 , and w 33 , which are derived from a transmission spectrum of the blue color filter at the red wavelength, the green wavelength, and the blue wavelength, respectively.