Systems, methods, and media for single photon depth imaging with improved efficiency using compressive histograms
In accordance with some embodiments, systems, methods, and media for single photon depth imaging with improved efficiency using compressive histograms are provided. In some embodiments, the system comprises: a light source; a detector configured to detect arrival of individual photons; a processor programmed to: detect a photon arrival; determine a time bin i of the photon arrival in a range from 1 to N a total number of time bins; update a compressed histogram comprising K stored values representing bins of the compressed histogram based on K values in a code word represented by an i th column of a coding matrix C having dimension K×N, with each column different than each other column, and each column corresponds to a single time bin i; and estimate a depth value based on the K values.
1 . A system for determining a depth in a scene, comprising:
a light source;
a detector configured to detect arrival of individual photons;
at least one processor that is programmed to:
(a) detect, based on a signal from the detector, a photon arrival;
(b) determine a time bin t associated with the photon arrival, wherein the time bin is in a range from 1 to N where N is a total number of time bins;
(c) update a compressed histogram comprising K stored values representing bins of the compressed histogram based on K values in a code word represented by an i th column of a coding matrix C having dimension K×N,
wherein each column of the coding matrix is different than each other column of the coding matrix, and each column corresponds to a single time bin i; and
(d) estimate a depth value based on the K values.
2 . The system of claim 1 , wherein the detector comprises a single photon avalanche diode (SPAD).
3 . The system of claim 1 , wherein the coding matrix is a Gray coding matrix.
4 . The system of claim 3 , wherein the Gray coding matrix is a binary Gray coding matrix.
5 . The system of claim 1 , wherein the coding matrix is a Gray-based Fourier coding matrix.
6 . The system of claim 1 , wherein the coding matrix is a Fourier-based Gray coding matrix.
7 . The system of claim 1 , wherein the at least one processor is further programmed to:
estimate the depth value using the following:
t
z
∝
arg
max
i
C
:
ι
h
-
mean
(
C
:
ι
h
)
C
:
,
ι
h
-
mean
(
C
:
,
ι
h
)
B
ˆ
-
mean
(
B
ˆ
)
B
ˆ
-
mean
(
B
ˆ
)
where t z is a time-of-flight corresponding to the depth value, C :,i h is the i th column of C h , and C h is the coding matrix C with each row convolved with an impulse response function h of the system used to detect the photon at (i), and {circumflex over (B)} is the compressed histogram.
8 . The system of claim 1 , wherein (a) to (c) are performed by circuitry that is implemented on the same chip as the detector.
9 . The system of claim 1 , the system further comprising an image sensor comprising an array including a plurality of detectors configured to detect arrival of individual photons, including the detector,
wherein the at least one processor is further programmed to:
perform (a) to (d) for each of the plurality of detectors.
10 . The system of claim 1 , the system further comprising an image sensor comprising an array including a first plurality of detectors in a first area and a second plurality of detectors in a second area, each of the first plurality of detectors and the second plurality of detectors configured to detect arrival of individual photons, including the detector,
wherein the at least one processor is further programmed to:
perform (a) to (d) for each of the first plurality of detectors; and
perform (a) to (d) for each of the second plurality of detectors using a second coding matrix C 1 having a number of rows K 1 >K.
11 . The system of claim 1 , wherein the photon arrival is detected during a first frame, and
wherein the at least one processor is further programmed to:
repeat (a) to (d) for a second frame using a second coding matrix C 1 having a number of rows K 1 >K.
12 . The system of claim 1 , wherein N is at least 1024, and K is no greater than 64.
13 . A method for determining a depth in a scene, comprising:
(a) detecting, based on a signal from a detector configured to detect arrival of individual photons, a photon arrival;
(b) determining a time bin t associated with the photon arrival, wherein the time bin is in a range from 1 to N where N is a total number of time bins;
(c) updating a compressed histogram comprising K stored values representing bins of the compressed histogram based on K values in a code word represented by an i th column of a coding matrix C having dimension K×N,
wherein each column of the coding matrix is different than each other column of the coding matrix, and each column corresponds to a single time bin i; and
(d) estimating a depth value based on the K values.
14 . The method of claim 13 , wherein the detector comprises a single photon avalanche diode (SPAD).
15 . The method of claim 13 , wherein the coding matrix is a Gray coding matrix.
16 . The method of claim 13 , wherein estimating the depth value comprises using the following:
t
z
∝
arg
max
i
C
:
ι
h
-
mean
(
C
:
ι
h
)
C
:
,
ι
h
-
mean
(
C
:
,
ι
h
)
B
ˆ
-
mean
(
B
ˆ
)
B
ˆ
-
mean
(
B
ˆ
)
where t z is a time-of-flight corresponding to the depth value, C :,i h is the i th column of C h , and C h is the coding matrix C with each row convolved with an impulse response function h of the system used to detect the photon at (i), and {circumflex over (B)} is the compressed histogram.
17 . The method of claim 13 , further comprising performing (a) to (d) for each of a plurality of detectors configured to detect arrival of individual photons, wherein the plurality of detectors and the detector are included in an image sensor comprising an array of detectors.
18 . The method of claim 13 , further comprising:
performing (a) to (d) for each of a first plurality of detectors of an image sensor comprising an array including the first plurality of detectors in a first area and a second plurality of detectors in a second area, each of the first plurality of detectors and the second plurality of detectors configured to detect arrival of individual photons, including the detector; and
performing (a) to (d) for each of the second plurality of detectors using a second coding matrix C 1 having a number of rows K 1 >K.
19 . A non-transitory computer readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for determining a depth in a scene, comprising:
(a) detecting, based on a signal from a detector configured to detect arrival of individual photons, a photon arrival;
(b) determining a time bin t associated with the photon arrival, wherein the time bin is in a range from 1 to N where N is a total number of time bins;
(c) updating a compressed histogram comprising K stored values representing bins of the compressed histogram based on K values in a code word represented by an i th column of a coding matrix C having dimension K×N,
wherein each column of the coding matrix is different than each other column of the coding matrix, and each column corresponds to a single time bin i; and
(d) estimating a depth value based on the K values.
20 . The non-transitory computer-readable medium of claim 19 , wherein the detector comprises a single photon avalanche diode (SPAD).