Systems and methods for obtaining color imagery using single photon avalanche diodes

1. A system for obtaining color imagery using single photon avalanche diodes (SPADs), the system comprising:

a SPAD array comprising a plurality of SPAD pixels, each of the plurality of SPAD pixels comprising a respective color filter positioned thereover;

one or more processors; and

one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to:

capture an image frame using the SPAD array;

generate a filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame, the at least one preceding image frame being captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame wherein performing the temporal filtering operation comprises:

generating a plurality of demultiplexed image frames by demultiplexing the image frame, each of the plurality of demultiplexed image frames being associated with a respective color channel;

accessing a plurality of demultiplexed preceding image frames, the plurality of demultiplexed preceding image frames being generated by demultiplexing the at least one preceding image frame, the plurality of demultiplexed preceding image frames comprising, for each particular demultiplexed image frame of the plurality of demultiplexed image frames, a corresponding demultiplexed preceding image frame associated with a same color channel as the particular demultiplexed image frame; and

generating a plurality of temporally filtered demultiplexed image frames by, for each particular demultiplexed image frame of the plurality of demultiplexed image frames:

generating a corresponding aligned demultiplexed preceding image frame by using motion data associated with the SPAD array to align the corresponding demultiplexed preceding image frame with the particular demultiplexed image frame; and

compositing the particular demultiplexed image frame with the corresponding aligned demultiplexed preceding image frame; and

after performing the temporal filtering operation, generate a color image by demosaicing the filtered image.

2. The system of claim 1 , wherein the respective color filters covering the plurality of SPAD pixels are arranged in a Bayer pattern.

3. The system of claim 1 , wherein the at least one preceding image frame comprises at least one raw image frame on which no demosaicing has been performed.

4. The system of claim 1 , wherein performing the temporal filtering operation comprises:

generating an aligned at least one preceding image frame by using motion data associated with the SPAD array to spatially align the at least one preceding image frame with the image frame; and

compositing the image frame with the aligned at least one preceding image frame.

5. The system of claim 1 , wherein generating the filtered image comprises multiplexing the plurality of temporally filtered demultiplexed image frames.

6. The system of claim 1 , wherein performing the temporal filtering operation comprises applying a joint bilateral filter to the image frame and the at least one preceding image frame.

7. The system of claim 6 , wherein performing the temporal filtering operation comprises applying the joint bilateral filter to the image frame and a subsequent image frame, the subsequent image frame being captured at a timepoint that is temporally subsequent to the timepoint associated with the image frame.

8. The system of claim 1 , wherein the instructions are executable by the one or more processors to further configure the system to perform a gamma correction operation on the filtered image prior to demosaicing the filtered image.

9. The system of claim 1 , wherein:

the system further comprises a display, and

the instructions are executable by the one or more processors to further configure the system to display the color image on the display.

10. The system of claim 1 , wherein:

the system further comprises a display, and

the instructions are executable by the one or more processors to further configure the system to:

generate a parallax corrected color image; and

display the parallax corrected color image on the display.

11. The system of claim 1 , wherein the instructions are executable by the one or more processors to further configure the system to perform a dark current compensation operation on the filtered image prior to demosaicing the filtered image.

12. A method for obtaining color imagery using single photon avalanche diodes (SPADs), the method comprising:

capturing an image frame using a SPAD array, the SPAD array comprising a plurality of SPAD pixels, each of the plurality of SPAD pixels comprising a respective color filter positioned thereover;

generating a filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame, the at least one preceding image frame being captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame wherein performing the temporal filtering operation comprises:

generating a plurality of demultiplexed image frames by demultiplexing the image frame, each of the plurality of demultiplexed image frames being associated with a respective color channel;

accessing a plurality of demultiplexed preceding image frames, the plurality of demultiplexed preceding image frames being generated by demultiplexing the at least one preceding image frame, the plurality of demultiplexed preceding image frames comprising, for each particular demultiplexed image frame of the plurality of demultiplexed image frames, a corresponding demultiplexed preceding image frame associated with a same color channel as the particular demultiplexed image frame; and

generating a plurality of temporally filtered demultiplexed image frames by, for each particular demultiplexed image frame of the plurality of demultiplexed image frames:

generating a corresponding aligned demultiplexed preceding image frame by using motion data associated with the SPAD array to align the corresponding demultiplexed preceding image frame with the particular demultiplexed image frame; and

compositing the particular demultiplexed image frame with the corresponding aligned demultiplexed preceding image frame; and

after performing the temporal filtering operation, generating a color image by demosaicing the filtered image.

13. One or more hardware storage devices that store instructions that are executable by one or more processors of a system to configure the system to:

capture an image frame using a single photon avalanche diode (SPAD) array, the SPAD array comprising a plurality of SPAD pixels, each of the plurality of SPAD pixels comprising a respective color filter positioned thereover;

generate a filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame, the at least one preceding image frame being captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame wherein performing the temporal filtering operation comprises:

generating a plurality of demultiplexed image frames by demultiplexing the image frame, each of the plurality of demultiplexed image frames being associated with a respective color channel;

accessing a plurality of demultiplexed preceding image frames, the plurality of demultiplexed preceding image frames being generated by demultiplexing the at least one preceding image frame, the plurality of demultiplexed preceding image frames comprising, for each particular demultiplexed image frame of the plurality of demultiplexed image frames, a corresponding demultiplexed preceding image frame associated with a same color channel as the particular demultiplexed image frame; and

generating a plurality of temporally filtered demultiplexed image frames by, for each particular demultiplexed image frame of the plurality of demultiplexed image frames:

generating a corresponding aligned demultiplexed preceding image frame by using motion data associated with the SPAD array to align the corresponding demultiplexed preceding image frame with the particular demultiplexed image frame; and

compositing the particular demultiplexed image frame with the corresponding aligned demultiplexed preceding image frame; and

after performing the temporal filtering operation, generate a color image by demosaicing the filtered image.