Systems and methods for dark current compensation in single photon avalanche diode imagery
A system for dark current compensation in SPAD imagery is configurable to capture an image frame with the SPAD array and generate a temporally filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame. The at least one preceding image frame is captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame. The system is also configurable to obtain a dark current image frame. The dark current image frame includes data indicating one or more SPAD pixels of the plurality of SPAD pixels that detect an avalanche event without detecting a corresponding photon. The system is also configurable to generate a dark current compensated image by performing a subtraction operation on the temporally filtered image or the image frame based on the dark current image frame.
1. A system for compensating for dark current, the system comprising:
an image sensor array comprising a plurality of image sensor pixels;
one or more processors; and
one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to:
capture an image frame with the image sensor array;
generate a temporally filtered image by performing a temporal filtering operation based on the image frame and at least one preceding image frame, the at least one preceding image frame having been captured by the image sensor array at a timepoint that temporally precedes a timepoint associated with the image frame;
obtain a dark current factor indicating an amount of dark current associated with the image frame;
select a dark current image frame, the dark current image frame comprising data indicating one or more image sensor pixels of the plurality of image sensor pixels that detect an avalanche event without detecting a corresponding photon, wherein the dark current image frame is one of a plurality of dark current image frames captured while the image sensor array is covered to prevent photons from reaching the image sensor array, wherein the dark current image frame is selected from the plurality of dark current image frames based on the dark current factor; and
generate a dark current compensated image by performing a subtraction operation on the temporally filtered image based on the dark current image frame.
2. The system of claim 1 , wherein the temporal filtering operation comprises:
generating at least one aligned preceding image frame by using motion data associated with the image sensor array to spatially align the at least one preceding image frame with the image frame; and
compositing the image frame with the aligned at least one preceding image frame.
3. The system of claim 1 , wherein the temporal filtering operation is based on an optical flow estimation generated based on the at least one preceding image frame and the image frame.
4. The system of claim 1 , wherein:
each of the plurality of dark current image frames is associated with a different temperature or range of temperatures, and
the dark current factor is based on a runtime temperature detected in association with the capturing of the image frame, such that the selection of the dark current image frame from the plurality of dark current image frames is based on the runtime temperature.
5. The system of claim 4 , wherein the instructions are executable by the one or more processors to configure the system to generate an interpolated dark current image frame based on the runtime temperature and at least two dark current image frames of the plurality of dark current image frames, and wherein the dark current image frame used for generating the dark current compensated image comprises the interpolated dark current image frame.
6. The system of claim 1 , wherein the plurality of image sensor pixels of the image sensor array comprises a plurality of unexposed image sensor pixels that are covered during the capturing of the dark current image frame and during the capturing of the image frame.
7. The system of claim 6 , wherein the dark current factor is based on (i) a quantity of avalanche events detected by the plurality of unexposed image sensor pixels during the capturing of the dark current image frame and (ii) a quantity of avalanche events detected by the plurality of unexposed image sensor pixels during the capturing of the image frame.
8. The system of claim 7 , wherein the subtraction operation comprises:
generating a scaled dark current image frame by modifying the dark current image frame using the dark current factor; and
subtracting the scaled dark current image frame from the image frame.
9. The system of claim 8 , wherein the dark current factor is determined by comparing (i) an average intensity detected by the plurality of unexposed image sensor pixels during the capturing of the dark current image frame and (ii) an average intensity detected by the plurality of unexposed image sensor pixels during the capturing of the image frame.
10. The system of claim 8 , wherein:
the dark current factor comprises a plurality of dark current factor components,
each dark current factor component is associated with a respective subset of unexposed image sensor pixels of the plurality of unexposed image sensor pixels, and
generating the scaled dark current image frame comprises using each of the plurality of dark current factor components to modify corresponding regions of the dark current image frame.
11. The system of claim 1 , wherein the instructions are executable by the one or more processors to further configure the system to perform a pixel binning operation on the dark current compensated image.
12. The system of claim 1 , wherein the instructions are executable by the one or more processors to further configure the system to apply a median filter to the dark current compensated image.
13. A system for compensating for dark current, the system comprising:
an image sensor array comprising a plurality of image sensor pixels;
one or more processors; and
one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to:
capture an image frame with the image sensor array;
generate a temporally filtered image by performing a temporal filtering operation based on the image frame and at least one preceding image frame, the at least one preceding image frame having been captured by the image sensor array at a timepoint that temporally precedes a timepoint associated with the image frame;
obtain a dark current factor indicating an amount of dark current associated with the image frame;
obtain a dark current image frame, the dark current image frame comprising data indicating one or more image sensor pixels of the plurality of image sensor pixels that detect an avalanche event without detecting a corresponding photon, wherein the dark current image frame is based on one or more image frames captured while the image sensor array is covered to prevent photons from reaching the image sensor array, wherein the plurality of image sensor pixels of the image sensor array comprises a plurality of unexposed image sensor pixels that are covered during the capturing of the dark current image frame and during the capturing of the image frame, wherein the dark current factor is based on (i) a quantity of avalanche events detected by the plurality of unexposed image sensor pixels during the capturing of the dark current image frame and (ii) a quantity of avalanche events detected by the plurality of unexposed image sensor pixels during the capturing of the image frame; and
generate a dark current compensated image by performing a subtraction operation on the temporally filtered image based on the dark current image frame and the dark current factor.
14. The system of claim 13 , wherein the temporal filtering operation comprises:
generating at least one aligned preceding image frame by using motion data associated with the image sensor array to spatially align the at least one preceding image frame with the image frame; and
compositing the image frame with the aligned at least one preceding image frame.
15. The system of claim 13 , wherein the temporal filtering operation is based on an optical flow estimation generated based on the at least one preceding image frame and the image frame.
16. The system of claim 13 , wherein the subtraction operation comprises:
generating a scaled dark current image frame by modifying the dark current image frame using the dark current factor; and
subtracting the scaled dark current image frame from the image frame.
17. The system of claim 16 , wherein the dark current factor is determined by comparing (i) an average intensity detected by the plurality of unexposed image sensor pixels during the capturing of the dark current image frame and (ii) an average intensity detected by the plurality of unexposed image sensor pixels during the capturing of the image frame.
18. The system of claim 16 , wherein:
the dark current factor comprises a plurality of dark current factor components,
each dark current factor component is associated with a respective subset of unexposed image sensor pixels of the plurality of unexposed image sensor pixels, and
generating the scaled dark current image frame comprises using each of the plurality of dark current factor components to modify corresponding regions of the dark current image frame.
19. The system of claim 13 , wherein the instructions are executable by the one or more processors to further configure the system to perform a pixel binning operation on the dark current compensated image.
20. A method for compensating for dark current, the method comprising:
capturing an image frame with an image sensor array comprising a plurality of image sensor pixels;
generating a temporally filtered image by performing a temporal filtering operation based on the image frame and at least one preceding image frame, the at least one preceding image frame having been captured by the image sensor array at a timepoint that temporally precedes a timepoint associated with the image frame;
obtaining a dark current factor indicating an amount of dark current associated with the image frame;
selecting a dark current image frame, the dark current image frame comprising data indicating one or more image sensor pixels of the plurality of image sensor pixels that detect an avalanche event without detecting a corresponding photon, wherein the dark current image frame is one of a plurality of dark current image frames captured while the image sensor array is covered to prevent photons from reaching the image sensor array, wherein the dark current image frame is selected from the plurality of dark current image frames based on the dark current factor; and
generating a dark current compensated image by performing a subtraction operation on the temporally filtered image based on the dark current image frame.