Methods and apparatus for transient light imaging
In illustrative implementations of this invention, multi-path analysis of transient illumination is used to reconstruct scene geometry, even of objects that are occluded from the camera. An ultrafast camera system is used. It comprises a photo-sensor (e.g., accurate in the picosecond range), a pulsed illumination source (e.g. a femtosecond laser) and a processor. The camera emits a very brief light pulse that strikes a surface and bounces. Depending on the path taken, part of the light may return to the camera after one, two, three or more bounces. The photo-sensor captures the returning light bounces in a three-dimensional time image I(x,y,t) for each pixel. The camera takes different angular samples from the same viewpoint, recording a five-dimensional STIR (Space Time Impulse Response). A processor analyzes onset information in the STIR to estimate pairwise distances between patches in the scene, and then employs isometric embedding to estimate patch coordinates.
1. Apparatus comprising, in combination,
an illumination source for emitting pulses of light toward a scene,
a sensor
for measuring the intensity of light from said scene, in such a manner that
the measured intensity at a pixel of said sensor from a pulse of light varies over time due to different paths that said light travels between said illumination source and said sensor, and
said sensor takes said measurements with respect to a plurality of said pulses, which plurality of pulses are not all in the same direction, and
for outputting signals indicative of data gathered in said measurements, and
one or more processors for processing said data to calculate positions of at least two points in said scene, based, at least in part, on different amounts of time it takes light to travel different paths.
2. The apparatus of claim 1 , wherein said illumination source is a pulsed laser.
3. The apparatus of claim 1 , wherein said apparatus is adapted for calculating the position of at least one point in said scene, which point is occluded from direct view of said illumination source and said sensor.
4. The apparatus of claim 1 , wherein said apparatus is adapted for calculating the distance to a substantially specular surface that does not reflect light directly back to said sensor.
5. The apparatus of claim 1 , wherein said illumination source and said sensor are housed in the same device.
6. The apparatus of claim 1 , wherein said sensor is directionally-sensitive.
7. The apparatus of claim 1 , wherein said sensor comprises an array of photosensors.
8. The apparatus of claim 1 , wherein said sensor is adapted to be able to take a measurement of light intensity at least as often as once every 50 picoseconds.
9. The apparatus of claim 1 , wherein said sensor is adapted to be able to take a measurement of light intensity at least as often as once every nanosecond.
10. The apparatus of claim 1 , wherein said data gathered by said sensor comprises an impulse response, the dimensions of which impulse response relate at least to pixel position, direction of light pulse, and time.
11. The apparatus of claim 10 , wherein said one or more processors use onset information in said impulse response to calculate pairwise distances between points in said scene.
12. The apparatus of claim 11 , wherein said one or more processors employ isometric embedding to calculate the position of at least two points in said scene.
13. A method comprising the following steps, in combination:
emitting a directionally-varying pulsed signal toward an environment,
using a sensor to take time samples of the intensity of the signal reflected back directly or indirectly to said sensor from said pulses, and
using one or more processors to calculate geometry of said scene, based at least in part on the different amounts of time it takes for a signal to travel different paths;
wherein at least one of said processors uses onset information to calculate pairwise distances between points in said scene, and wherein at least one of said processors uses isometric embedding to calculate the position of at least two points in said scene.
14. The method of claim 13 , wherein said sensor measures light intensity at least once every 50 picoseconds.
15. A process comprising, in combination:
emitting pulses of light toward a scene,
using a directionally-sensitive sensor to take a plurality of time samples of light reflecting back from said scene, in such a way as to record a multi-dimensional impulse response, the dimensions of which impulse response relate at least to pixel position, direction of light pulse, and time,
using onset information in said impulse response to calculate pairwise distances between points in said scene, and
using isometric embedding to calculate the position of at least two points in said scene.
16. The process of claim 15 , wherein said pulses of light are emitted by a laser.
17. The process of claim 16 , wherein the position of at least one point in said scene is calculated, which point is occluded from direct view of said sensor.
18. The process of claim 15 , wherein a time sample captures changes in light intensity at a pixel as light from a pulse travels back to said pixel over different paths.