3D image reconstruction based on lensless compressive image acquisition
The present disclosure generally discloses a three-dimensional (3D) image reconstruction capability. The 3D image reconstruction capability may be configured to support reconstruction of a 3D image of a scene. The 3D image reconstruction capability may be configured to support reconstruction of a 3D image of a scene based on lensless compressive image acquisition performed using a lensless compressive camera having a single aperture and a set of multiple sensors. The reconstructed 3D image of a scene may include (1) image data indicative of a set of multiple two-dimensional (2D) images reconstructed based on the set of multiple sensors of the lensless compressive camera (which may be represented as images) and (2) depth information indicative of depths at points or areas of an overlapping portion of the multiple images reconstructed based on the set of multiple sensors of the lensless compressive camera (which may be represented as a depth map).
1. An apparatus, comprising:
a processor and a memory communicatively connected to the processor, the processor configured to:
receive camera geometry information associated with a lensless compressive camera comprising a programmable aperture and a sensor plane including a pair of sensors, wherein the camera geometry information comprises an inter-sensor distance between the sensors and a distance between the programmable aperture and the sensor plane;
obtain reconstructed image data indicative of a pair of reconstructed images reconstructed based on respective sets of compressive measurements associated with the respective pair of sensors;
determine, based on the reconstructed image data, disparity information associated with a common image portion that is common to the pair of reconstructed images; and
determine, based on the camera geometry information and the disparity information associated with the common image portion, depth information associated with the common image portion.
2. The apparatus of claim 1 , wherein the processor is configured to:
associate the reconstructed image data and the depth information to provide thereby a reconstructed three-dimensional (3D) image.
3. The apparatus of claim 1 , wherein the camera geometry information further comprises an element size associated with programmable elements of the programmable aperture.
4. The apparatus of claim 1 , wherein, to obtain the reconstructed image data, the processor is configured to: receive the reconstructed image data from a storage device.
5. The apparatus of claim 4 , wherein the processor is configured to receive the reconstructed image data from the storage device via a communication network.
6. The apparatus of claim 1 , wherein, to obtain the reconstructed image data, the processor is configured to:
receive a first set of compressive measurements associated with a first sensor of the pair of sensors and generate, based on the first set of compressive measurements, first reconstructed image data indicative of a first reconstructed image of the pair of reconstructed images; and
receive a second set of compressive measurements associated with a second sensor of the pair of sensors and generate, based on the second set of compressive measurements, second reconstructed image data indicative of a second reconstructed image of the pair of reconstructed images.
7. The apparatus of claim 6 , wherein the processor is configured to receive the first set of compressive measurements and the second set of compressive measurements from the lensless compressive camera via a communication network.
8. The apparatus of claim 1 , wherein the processor is configured to:
identify the common image portion that is common to the pair of reconstructed images.
9. The apparatus of claim 8 , wherein, to identify the common image portion, the processor is configured to:
process the pair of reconstructed images, based on at least one of a block matching algorithm or a feature matching algorithm, to identify the common image portion.
10. The apparatus of claim 1 , wherein the depth information associated with the common image portion is proportional to the disparity information associated with the common image portion.
11. The apparatus of claim 1 , wherein, to determine the depth information associated with the common image portion, the processor is configured to:
for each of a plurality of regions of the common image portion:
determine, from the disparity information associated with the common image portion, a disparity value associated with the region of the common image portion; and
determine a depth value associated with the region of the common image portion based on the camera geometry information and the disparity value associated with the region of the common image portion.
12. The apparatus of claim 11 , wherein the region of the common image portion comprise a pixel or a group of pixels.
13. The apparatus of claim 11 , wherein the processor is configured to determine the depth value associated with the region of the common image portion based on PD =[( M 1 M 2 ] DS 0 )/( S 1 S 2 − M 1 M 2 )], wherein PD is the depth value, M 1 M 2 is the disparity value, S 1 S 2 is the inter-sensor distance between the sensors, and DS 0 is the distance between the programmable aperture and the sensor plane.
14. The apparatus of claim 1 , wherein the apparatus forms part of the lensless compressive camera.
15. The apparatus of claim 1 , wherein the apparatus is configured to communicate with the lensless compressive camera via a communication network or is configured to form part of a device configured to communicate with the lensless compressive camera via a communication network.
16. A method, comprising:
receiving, by a processor, camera geometry information associated with a lensless compressive camera comprising a programmable aperture and a sensor plane including a pair of sensors, wherein the camera geometry information comprises an inter-sensor distance between the sensors and a distance between the programmable aperture and the sensor plane;
obtaining, by the processor, reconstructed image data indicative of a pair of reconstructed images reconstructed based on respective sets of compressive measurements associated with the respective pair of sensors;
determining, by the processor based on the reconstructed image data, disparity information associated with a common image portion that is common to the pair of reconstructed images; and
determining, by the processor based on the camera geometry information and the disparity information associated with the common image portion, depth information associated with the common image portion.
17. A lensless compressive camera, comprising:
a programmable aperture;
a pair of sensors disposed on a sensor plane and configured to detect light passing through the programmable aperture;
a memory configured to store camera geometry information associated with the lensless compressive camera, wherein the camera geometry information comprises an inter-sensor distance between the sensors and a distance between the programmable aperture and the sensor plane; and
a processor configured to:
determine reconstructed image data indicative of a pair of reconstructed images reconstructed based on respective sets of compressive measurements associated with the respective pair of sensors of the lensless compressive camera;
determine, based on the reconstructed image data, disparity information associated with a common image portion that is common to the pair of reconstructed images; and
determine, based on the camera geometry information and the disparity information associated with the common image portion, depth information associated with the common image portion.