Image demosaicing for hybrid optical sensor arrays

1. A method for an imaging device, comprising:

receiving, from a hybrid optical sensor array comprising n pixels, a first set of data for a scene captured by a first set of pixels and a second set of data for the scene captured by a second set of pixels, wherein:

the first set of pixels and second set of pixels total n pixels, where n is a positive integer;

the first set of pixels and second set of pixels comprise different numbers of pixels;

the first set of pixels is sensitive to infrared light; and

the second set of pixels comprises a first subset of pixels sensitive to red light, a second subset of pixels sensitive to green light, and a third subset of pixels sensitive to blue light;

demosaicing the first set of data based on at least the second set of data and demosaicing the second set of data based on at least the first set of data, such that a higher resolution data set is utilized to increase the resolution of a lower resolution data set; and

outputting a third set of data for the scene comprising an infrared image with n pixels, and outputting a fourth set of data for the scene comprising an RGB image with n pixels.

2. The method of claim 1 , wherein demosaicing the first set of data based on at least the second set of data comprises:

interpolating the first set of data independent of the second set of data; and

filtering the interpolated first set of data based on the second set of data.

3. The method of claim 2 , wherein the second set of pixels comprises a greater number of pixels than the first set of pixels, and wherein filtering the interpolated first set of data based on the second set of data further comprises:

filtering the interpolated first set of data based on guidance image weights derived from the second set of data.

4. The method of claim 1 , wherein demosaicing the second set of data based on at least the first set of data comprises:

interpolating the second set of data independent of the first set of data; and

filtering the interpolated second set of data based on the first set of data.

5. The method of claim 4 , wherein the first set of pixels comprises a greater number of pixels than the second set of pixels, and wherein filtering the interpolated second set of data based on the first set of data further comprises:

filtering the interpolated second set of data based on guidance image weights derived from the first set of data.

6. The method of claim 1 , wherein the demosaicing of the first set of data and the second set of data comprises:

jointly interpolating the first set of data and the second set of data; and

jointly re-interpolating the interpolated first set of data and the interpolated second set of data.

7. The method of claim 6 , wherein the second set of pixels comprises a greater number of pixels than the first set of pixels, and wherein jointly interpolating the first set of data and the second set of data comprises:

interpolating the first set of data based on guidance image weights derived from the second set of data.

8. The method of claim 6 , wherein the first set of pixels comprises a greater number of pixels than the second set of pixels, and wherein jointly interpolating the first set of data and the second set of data comprises:

interpolating the first set of data based on guidance image weights derived from the second set of data.

9. The method of claim 1 , further comprising:

generating depth information for the scene based on at least the third set of data.

10. The method of claim 1 , further comprising:

generating depth information for the scene based on at least the first set of data, and wherein demosaicing the second set of data based on at least the first set of data further comprises demosaicing the second set of data based on the generated depth information for the scene.

11. An imaging system, comprising:

a hybrid optical sensor array comprising n pixels, where n is a positive integer, then pixels comprising a first set of pixels and a second set of pixels, the first set of pixels comprising n/4 pixels sensitive to infrared light, the second set of pixels comprising 3n/4 pixels, including a first subset of pixels sensitive to red light, a second subset of pixels sensitive to green light, and a third subset of pixels sensitive to blue light;

a storage machine holding instructions executable by a logic machine to:

receive infrared light of a scene at the first set of pixels;

receive visible light of the scene at the second set of pixels;

generate a first RGB data set of the scene based on the received visible light;

generate a first infrared light data set of the scene based on the received infrared light;

demosaic the infrared data set of the scene based on spatial information derived from the first RGB data set of the scene;

output a second infrared light data set of the scene comprising n pixels.

12. The imaging system of claim 11 , wherein demosaicing the first infrared data set of the scene based on spatial information derived from the first RGB data set of the scene comprises:

interpolating the first infrared data set independent of the first RGB data set; and

filtering the interpolated infrared data set based on the first RGB data set.

13. The imaging system of claim 11 , wherein the storage machine further holds instructions executable by the logic machine to:

demosaic the first RGB data set of the scene based on the infrared data set of the scene; and

wherein demosaicing the first infrared data set of the scene and the first RGB data set of the scene comprises:

jointly interpolating the first infrared data set and the first RGB data set; and

jointly re-interpolating the interpolated infrared data set and the interpolated RGB data set.

14. The imaging system of claim 11 , further comprising:

a time-of-flight depth camera, and wherein the storage machine further holds instructions executable by the logic machine to:

generate two or more infrared light data sets of the scene based on the received infrared light;

demosaic each infrared data set of the scene based on the first RGB data set of the scene, each infrared data set weighted equally during demosaicing; and

output depth values for the scene comprising n pixels based on the demosaiced infrared data sets.

15. The imaging system of claim 11 , further comprising:

a structured light depth camera, and wherein the storage machine further holds instructions executable by the logic machine to:

generate depth values for the scene based on the first infrared light data set of the scene;

demosaic the first RGB data set of the scene based on the generated depth values for the scene;

demosaic the generated depth values for the scene based on spatial information derived from the first RGB data set of the scene; and

output depth values for the scene comprising n pixels based on the demosaiced generated depth values.

16. An imaging system, comprising:

a hybrid optical sensor array comprising n pixels, where n is a positive integer, then pixels comprising a first set of pixels and a second set of pixels, the first set of pixels comprising 3n/4 pixels sensitive to infrared light, the second set of pixels comprising n/4 pixels, including a first subset of pixels sensitive to red light, a second subset of pixels sensitive to green light, and a third subset of pixels sensitive to blue light;

a storage machine holding instructions executable by a logic machine to:

receive infrared light of a scene at the first set of pixels;

receive visible light of the scene at the second set of pixels;

generate a first RGB data set of the scene based on the received visible light;

generate a first infrared light data set of the scene based on the received infrared light;

demosaic the first RGB data set of the scene based on guidance image weights derived from the infrared light data set of the scene; and

output a second RGB data set of the scene comprising n pixels.

17. The imaging system of claim 16 , wherein demosaicing the first RGB data set of the scene based on at least guidance image weights derived from the infrared light data set of the scene comprises:

interpolating the first RGB data set independent of the first infrared data set; and

filtering the interpolated RGB data set based on the infrared data set.

18. The imaging system of claim 16 , wherein the storage machine further holds instructions executable by the logic machine to:

demosaic the first infrared data set of the scene based on the RGB data set of the scene; and

wherein demosaicing the first infrared data set of the scene and the first RGB data set of the scene comprises:

jointly interpolating the first infrared data set and the first RGB data set; and

jointly re-interpolating the interpolated infrared data set and the interpolated RGB data set.

19. The imaging system of claim 16 , further comprising:

a time-of-flight depth camera, and wherein the storage machine further holds instructions executable by the logic machine to:

generate two or more infrared light data sets of the scene based on the received infrared light;

demosaic each infrared data set of the scene based on the first RGB data set of the scene, each infrared data set weighted equally during demosaicing; and

output depth values for the scene comprising n pixels based on the demosaiced infrared data sets.

20. The imaging system of claim 16 , further comprising:

a structured light depth camera, and wherein the storage machine further holds instructions executable by the logic machine to:

generate depth values for the scene based on the first infrared light data set of the scene;

demosaic the first RGB data set of the scene based on guidance image weights derived from the generated depth values for the scene;

demosaic the generated depth values for the scene based on the first RGB data set of the scene; and

output depth values for the scene comprising n pixels based on the demosaiced generated depth values.