RGB-IR pixel pattern conversion via adaptive filtering

1 . An image processing pipeline, comprising:

an upstream component;

a pattern conversion component downstream with respect to the upstream component in the image processing pipeline; and

a downstream component relative to the pattern conversion component;

wherein the pattern conversion component is configured to:

obtain red green blue (RGB)-infrared (IR) pixel data produced by the upstream component of the image processing pipeline;

convert the RGB-IR pixel data into RGB pixel data and IR pixel data using an adaptive filter; and

supply the RGB pixel data and IR pixel data to the downstream component of the image processing pipeline;

wherein to convert the RGB-IR pixel data into the RGB pixel data and the IR pixel data, the adaptive filter is configured to, for each pixel of the RGB-IR pixel data:

select a weight from weights generated for the pixel based on a type of conversion operation including a remosaicing operation, an upsampling operation, and a decontamination operation; and

apply the weight to the pixel, resulting in an interpolated value for the pixel for use in the conversion operation.

2 . The image processing pipeline of claim 1 , wherein the adaptive filter is further configured to:

for each pixel of the RGB-IR pixel data:

identify one or more features of the pixel based on a context of the pixel, wherein the one or more features comprise a location feature indicative of a location of the pixel with respect to other pixels and a brightness feature indicative of a brightness of the location; and

generate the weights based on the features.

3 . The image processing pipeline of claim 1 , wherein the adaptive filter is further configured to convert the RGB-IR pixel data into the RGB pixel data and the IR pixel data using the interpolated values of the pixels of the RGB-IR pixel data.

4 . The image processing pipeline of claim 2 , wherein the context of the pixel comprises a characteristic of the pixel and a characteristic of one or more neighboring pixels, with respect to the pixel, of the RGB-IR pixel data.

5 . The image processing pipeline of claim 1 , wherein to apply the weight, the adaptive filter is configured to obtain the weight from a look-up table and perform the conversion operation with the pixel.

6 . The image processing pipeline of claim 5 , wherein a color channel of the pixel includes one among RGB or IR.

7 . The image processing pipeline of claim 2 , wherein the adaptive filter is further configured to:

for each pixel of the RGB-IR pixel data:

identify an IR contamination value of the pixel; and

subtract the IR contamination value from the interpolated value of the pixel.

8 . The image processing pipeline of claim 1 , wherein the adaptive filter comprises a convolutional neural network.

9 . A pattern conversion component in an image processing pipeline, comprising:

an interface configured to communicate with upstream components and downstream components of the image processing pipeline relative to the pattern conversion component; and

circuitry coupled to the interface and configured to:

obtain, via the interface, red green blue (RGB)-infrared (IR) pixel data produced by an upstream component of the image processing pipeline;

convert the RGB-IR pixel data into RGB pixel data and IR pixel data using an adaptive filter; and

supply, via the interface, the RGB pixel data and the IR pixel data to a downstream component of the image processing pipeline;

wherein to convert the RGB-IR pixel data into the RGB pixel data and the IR pixel data, the circuitry is configured to, for each pixel of the RGB-IR pixel data:

select a weight from weights generated for the pixel based on a type of conversion operation including a remosaicing operation, an upsampling operation, and a decontamination operation; and

apply the weight to the pixel, resulting in an interpolated value for the pixel for use in the conversion operation.

10 . The pattern conversion component of claim 9 , wherein the circuitry is further configured to:

for each pixel of the RGB-IR pixel data:

identify one or more features of the pixel based on a context of the pixel, wherein the one or more features comprise a location feature indicative of a location of the pixel with respect to other pixels and a brightness feature indicative of a brightness of the location; and

generate the weights based on the features.

11 . The pattern conversion component of claim 9 , wherein the circuitry is further configured to convert the RGB-IR pixel data into the RGB pixel data and the IR pixel data using the interpolated values of the pixels of the RGB-IR pixel data.

12 . The pattern conversion component of claim 10 , wherein the context of the pixel comprises a characteristic of the pixel and a characteristic of one or more neighboring pixels with respect to the pixel.

13 . The pattern conversion component of claim 9 , wherein to apply the weight, the circuitry is configured to obtain the weight from a look-up table and perform the conversion operation with the pixel.

14 . The pattern conversion component of claim 13 , wherein a color channel of the pixel includes one among RGB or IR.

15 . The pattern conversion component of claim 9 , wherein the circuitry is further configured to:

for each pixel of the RGB-IR pixel data:

identify an IR contamination value of the pixel; and

subtract the IR contamination value from the interpolated value of the pixel.

16 . The pattern conversion component of claim 9 , wherein the adaptive filter comprises a convolutional neural network.

17 . A method comprising:

obtaining, by a pattern conversion component of an image processing pipeline, red green blue (RGB)-infrared (IR) pixel data produced by a component of the image processing pipeline upstream relative to the pattern conversion component;

converting, by the pattern conversion component, the RGB-IR pixel data into RGB pixel data and IR pixel data by:

identifying, by the pattern conversion component, features of pixels of the RGB-IR pixel data based on a context of the pixels;

calculating, by the pattern conversion component, weights for the pixels based on the features; and

applying, by the pattern conversion component, a weight to each of the pixels, wherein the applied weight is selected based on a color channel of a given pixel and a type of conversion operation applied to the given pixel, and wherein the type of operation includes one or more of a remosaicing operation, an upsampling operation, and a decontamination operation; and

supplying, by the pattern conversion component, the RGB pixel data and IR pixel data to a downstream component of the image processing pipeline relative to the pattern conversion component.

18 . The method of claim 17 , wherein the context comprises a characteristic of the RGB-IR pixel data at a location associated with a given pixel and a characteristic of neighboring pixels of the RGB-IR pixel data with respect to the given pixel.

19 . The method of claim 17 , wherein the applied weight is further selected based on a desired output pixel array including either the RGB pixel data or the IR pixel data.

20 . The method of claim 18 , wherein the method further comprises:

identifying IR contamination values of the pixels of the RGB-IR pixel data; and

subtracting the IR contamination values from the pixels of the RGB pixel data.