Image demosaicer and method

1. A method for demosaicing a raw image captured by an image sensor that includes a pixel array and a Bayer-pattern color-filter array (CFA) having a plurality of red color filters CF r , green color filters CF g , and blue color filters CF b each covering a respective one of a plurality of pixel sub-arrays of the pixel array, the pixel array including a plurality of pixels p(x,y) α located at a respective one of a plurality of coordinates (x,y) α , and beneath a color filter CF α , and each generating a respective one of a plurality of primary pixel-values B α (x,y) α , where α is one of r, g, and b, the method comprising:

horizontally-interpolating a green channel, of the raw image, formed of primary pixel-values B g (x,y) g of the plurality of primary pixel-values B α (x,y) α , to yield a horizontally-interpolated green channel that includes both primary pixel-values B g (x,y) g and non-primary pixel-values I gh (x,y) r,b ;

modifying each non-primary pixel-value I gh (x,y) r,b by a plurality of horizontally-neighboring non-primary pixel-values I gh (x,y) r,b , to yield a refined horizontally-interpolated green channel;

vertically-interpolating the green channel to yield a vertically-interpolated green channel that includes both primary pixel-values B g (x,y) g and non-primary pixel-values I gv (x,y) r,b ;

modifying each non-primary pixel-value I gv (x,y) r,b by a plurality of vertically-neighboring non-primary pixel-values I gv (x,y) r,b , to yield a refined vertically-interpolated green channel;

generating a full-resolution green channel from the refined horizontally-interpolated green channel, refined vertically-interpolated green channel, a horizontal-gradient of the horizontally-interpolated green channel, and a vertical-gradient of the vertically-interpolated green channel;

generating a full-resolution red channel by determining a plurality of non-primary red pixel-values I r (x t ,y t ) g,b , each corresponding to a respective one of the plurality of coordinates (x t ,y t ) g,b , from a local-red mean value of neighboring red primary pixel-values B r (x,y) r and the full-resolution green channel;

generating a full-resolution blue channel by determining a plurality of non-primary blue pixel-values I b (x t ,y t ) g,r , each corresponding to a respective one of the plurality of coordinates (x t ,y t ) g,r , from a local-blue mean value of neighboring blue primary pixel-values B b (x,y) b and the full-resolution green channel; and

combining the full-resolution red channel, the full-resolution green channel, and the full-resolution blue channel to yield a demosaiced raw image.

2. The method of claim 1 , both the horizontally-interpolating the green channel and the vertically-interpolating the green channel comprising linearly interpolating a logarithm of primary pixel-values B g (x,y) g .

3. The method of claim 1 , each of the plurality of pixel sub-arrays having dimensions m×n, the step of modifying each non-primary pixel-value I gh (x,y) r,b comprising:

for each pixel p(x,y) beneath a red color filter of a target pixel sub-array, of the plurality of pixel sub-arrays, located between (i) a pair of horizontally-neighboring source pixel sub-arrays each beneath respective red color filters, of the CFA and horizontally collinear with the target pixel sub-array,

determining a red horizontal-supplement H r (x,y) as a linear combination of B r (x,y) r , primary pixel-values B r (x,y 1 ) r of horizontally-adjacent pixels p(x,y 1 ) beneath the target pixel sub-array, and primary pixel-values B r (x,y 2 ) r of horizontally-collinear pixels p(x,y 2 ) beneath red color filters of the pair of horizontally-neighboring source pixel sub-arrays, where |y−y 1 |<n and |y−y 2 |≥n.

4. The method of claim 1 , each of the plurality of pixel sub-arrays having dimensions m×n, the step of modifying each non-primary pixel-value I gv (x,y) r,b , comprising:

for each pixel p(x,y) beneath a red color filter of a target pixel sub-array, of the plurality of pixel sub-arrays, located between a pair of vertically-neighboring source pixel sub-arrays each beneath a respective red color filter of the CFA and vertically collinear with the target pixel sub-array,

determining a red vertical-supplement V r (x,y) as a linear combination of B r (x,y) r , primary pixel-values B r (x 1 ,y) r of vertically-adjacent pixels p(x 1 ,y) beneath the target pixel sub-array, and primary pixel-values B r (x 2 ,y) r of vertically-collinear pixels p(x 2 ,y) beneath red color filters of the pair of vertically-neighboring source pixel sub-arrays, where |x−x 1 |<m and |x−x 2 |≥m.

5. The method of claim 1 , further comprising determining the horizontal-gradient and the vertical-gradient by:

subtracting the raw image from the refined horizontally-interpolated green channel, and

subtracting the raw image from the refined vertical-interpolated green channel.

6. The method of claim 1 , at least one of (i) generating the full-resolution red channel and (ii) generating the full-resolution blue channel including:

binning the full-resolution green channel, and

binning the raw image.

7. The method of claim 6 , the at least one of (i) generating the full-resolution red channel and (ii) generating the full-resolution blue channel including: computing a variance of the binned full-resolution green channel and a covariance of (a) the binned full-resolution green channel and (b) the binned raw image.

8. The method of claim 6 ,

binning of the full-resolution green channel yielding a binned green channel that includes a plurality of binned green pixel-values,

binning the raw image yielding a binned raw image that includes a plurality of binned red pixel-values based on the plurality of primary pixel-values B r and a plurality of binned blue pixel-values based on the plurality of primary pixel-values B b , the local-red mean value being an average of binned red pixel-values, the local-blue mean value being an average of binned blue pixel-values.

9. The method of claim 8 ,

each of the plurality of non-primary red pixel-values I r (x t ,y t ) g,b having a magnitude that increases in response to an increasing covariance of the average of binned red pixel-values and an average of binned green pixel-values; and

each of the plurality of non-primary blue pixel-values I b (x t ,y t ) g,r having a magnitude that increases in response to an increasing covariance of the average of binned blue pixel-values and the average of binned green pixel-values.

10. The method of claim 8 ,

each of the plurality of non-primary red pixel-values I r (x t ,y t ) g,b having a magnitude that decreases in response to an increasing variance of the average of binned green pixel-values; and

each of the plurality of non-primary blue pixel-values I b (x t ,y t ) g,r having a magnitude that decreases in response to an increasing variance of the average of binned green pixel-values.

11. An image demosaicer for demosaicing a raw image captured by an image sensor that includes a pixel array and a Bayer-pattern color-filter array (CFA) having a plurality of red color filters CF r , green color filters CF g , and blue color filters CF b each covering a respective one of a plurality of m×n pixel sub-arrays of the pixel array, the pixel array including a plurality of pixels p(x,y) α located at a respective one of a plurality of coordinates (x,y) α , and beneath a color filter CF α , and each generating a respective one of a plurality of primary pixel-values B α (x,y) α , where α is one of r, g, and b:

a memory storing non-transitory computer-readable instructions and adapted to store the raw image;

a microprocessor adapted to execute the instructions to:

horizontally-interpolate a green channel, of the raw image, formed of primary pixel-values B g (x,y) g of the plurality of primary pixel-values B α (x,y) α , to yield a horizontally-interpolated green channel that includes both primary pixel-values B g (x,y) g and non-primary pixel-values I gh (x,y) r,b ;

modify each non-primary pixel-value I gh (x,y) r,b by a plurality of horizontally-neighboring non-primary pixel-values I gh (x,y) r,b , to yield a refined horizontally-interpolated green channel;

vertically-interpolate the green channel to yield a vertically-interpolated green channel that includes both primary pixel-values B g (x,y) g and non-primary pixel-values I gv (x,y) r,b ;

modify each non-primary pixel-value I gv (x,y) r,b by a plurality of vertically-neighboring non-primary pixel-values I gv (x,y) r,b , to yield a refined vertically-interpolated green channel;

generate a full-resolution green channel from the refined horizontally-interpolated green channel, refined vertically-interpolated green channel, a horizontal-gradient of the horizontally-interpolated green channel, and a vertical-gradient of the vertically-interpolated green channel;

generate a full-resolution red channel by determining a plurality of non-primary red pixel-values I r (x t ,y t ) g,b , each corresponding to a respective one of the plurality of coordinates (x t ,y t ) g,b , from a local-red mean value of neighboring red primary pixel-values B r (x,y) r and the full-resolution green channel;

generate a full-resolution blue channel by determining a plurality of non-primary blue pixel-values I b (x t ,y t ) g,r , each corresponding to a respective one of the plurality of coordinates (x t ,y t ) g,r , from a local-blue mean value of neighboring blue primary pixel-values B b (x,y) b and the full-resolution green channel; and

combine the full-resolution red channel, the full-resolution green channel, and the full-resolution blue channel to yield a demosaiced raw image.

12. The image demosaicer of claim 11 , the microprocessor being further adapted to execute the instructions to, when both horizontally-interpolating the green channel and vertically-interpolating the green channel, linearly interpolate a logarithm of primary pixel-values B g (x,y).

13. The image demosaicer of claim 11 , each of the plurality of pixel sub-arrays having dimensions m×n, when restoring resolution to the horizontally-interpolated green channel and for each pixel p(x,y) beneath a red color filter of a target pixel sub-array, of the plurality of pixel sub-arrays, located between (i) a pair of horizontally-neighboring source pixel sub-arrays each beneath respective red color filters, of the CFA and horizontally collinear with the target pixel sub-array, the microprocessor being further adapted to execute the instructions to:

determine a red horizontal-supplement H r (x,y) as a linear combination of B r (x,y) r , primary pixel-values B r (x,y 1 ) r of horizontally-adjacent pixels p(x,y 1 ) beneath the target pixel sub-array, and primary pixel-values B r (x,y 2 ) r of horizontally-collinear pixels p(x,y 2 ) beneath red color filters of the pair of horizontally-neighboring source pixel sub-arrays, where |y−y 1 |<n and |y−y 2 |≥n.

14. The image demosaicer of claim 11 , each of the plurality of pixel sub-arrays having dimensions m×n, when restoring resolution to the vertically-interpolated green channel and for each pixel p(x,y) beneath a red color filter of a target pixel sub-array, of the plurality of pixel sub-arrays, located between (i) a pair of vertically-neighboring source pixel sub-arrays each beneath respective red color filters, of the CFA and vertically collinear with the target pixel sub-array, the microprocessor being further adapted to execute the instructions to:

determine a red vertical-supplement V r (x,y) as a linear combination of B r (x,y), primary pixel-values B r (x 1 ,y) of vertically-adjacent pixels p(x 1 ,y) beneath the target pixel sub-array, and primary pixel-values B r (x 2 ,y) of vertically-collinear pixels p (x 2 ,y) beneath red color filters of the pair of vertically-neighboring source pixel sub-arrays, where |x−x 1 |<m and |x−x 2 |≥m.

15. The image demosaicer of claim 11 , the microprocessor being further adapted to execute the instructions to determine the horizontal-gradient and the vertical-gradient by:

subtracting the raw image from the refined horizontally-interpolated green channel; and

subtracting the raw image from the refined vertical-interpolated green channel.

16. The image demosaicer of claim 11 , the microprocessor being further adapted to execute the instructions to, when at least one of (i) generating the full-resolution red channel and (ii) generating the full-resolution blue channel:

bin the full-resolution green channel, and

bin the raw image.

17. The image demosaicer of claim 16 , the microprocessor being further adapted to execute the instructions to, when generating a full-resolution red channel, compute a variance of the binned full-resolution green channel and a covariance of (a) the binned full-resolution green channel and (b) the binned raw image.

18. The image demosaicer of claim 16 ,

binning of the full-resolution green channel yielding a binned green channel that includes a plurality of binned green pixel-values,

binning the raw image yielding a binned raw image that includes a plurality of binned red pixel-values based on the plurality of primary pixel-values B r and a plurality of binned blue pixel-values based on the plurality of primary pixel-values B b , the local-red mean value being an average of binned red pixel-values, the local-blue mean value being an average of binned blue pixel-values.

19. The image demosaicer of claim 18 ,

each of the plurality of non-primary red pixel-values I r (x t ,y t ) g,b having a magnitude that increases in response to an increasing covariance of the average of binned red pixel-values and the average of binned green pixel-values; and

each of the plurality of non-primary blue pixel-values I b (x t ,y t ) g,r having a magnitude that increases in response to an increasing covariance of the average of binned blue pixel-values and the average of binned green pixel-values.

20. The image demosaicer of claim 18 ,

each of the plurality of non-primary red pixel-values I r (x t ,y t ) g,b having a magnitude that decreases in response to an increasing variance of the average of binned green pixel-values; and

each of the plurality of non-primary blue pixel-values I b (x t ,y t ) g,r having a magnitude that decreases in response to an increasing variance of the average of binned green pixel-values.