Methods circuits devices assemblies systems and functionally associated with computer executable code for image acquisition with depth estimation
Disclosed are methods, circuits, devices, systems and functionally associated computer executable code for image acquisition with depth estimation. According to some embodiments, there may be provided an imaging device including: (a) one or more imaging assemblies with at least one image sensor; (b) at least one structured light projector adapted to project onto a scene a multiresolution structured light pattern, which patterns includes multiresolution symbols or codes; and (3) image processing circuitry, dedicated or programmed onto a processor, adapted to identify multiresolution structured light symbols/codes within an acquired image of the scene.
1. A depth sensing system, comprising:
a projector configured to project a bi-dimensional structured light pattern onto a scene, wherein the bi-dimensional structured light pattern is comprised of a plurality of fine feature types, each one of the plurality of fine feature types is formed by a unique combination of a plurality of fine feature elements;
a digital memory device storing data with respect to the plurality of fine feature types, and further storing data with respect to a plurality of coarse feature types, wherein each coarse feature type from the plurality of coarse feature types is comprised of a unique combination of a plurality of coarse feature elements, wherein an appearance of a coarse feature element corresponds to a fusing of two or more fine feature elements;
a sensor configured to capture an image of a reflected portion of the projected bi-dimensional structured light pattern; and
at least one processor coupled to the sensor and to the digital memory device, the at least one processor being configured to:
decode the image of the reflected portion of the projected bi-dimensional structured light pattern to thereby assign depth values to points in the scene;
detect in a region of interest of the image a fine feature type from the plurality of fine feature types; and
detect in the image a coarse feature type from the plurality of coarse feature types.
2. The system according to claim 1 , wherein the plurality of coarse feature elements do not exist in the projected bi-dimensional structured light pattern.
3. The system according to claim 1 , wherein the plurality of coarse feature elements in the image result from projection or imaging conditions which cause respective fine feature elements to appear fused with one another in the image.
4. The system according to claim 3 , wherein the projection or imaging conditions include one or more of the following: object translucency, defocus blurring, and three-dimensional texture.
5. The system according to claim 1 , wherein the at least one processor is configured to downsample an area of the image and use coarse feature types for estimating depth values within an area of the scene which corresponds to the downsampled area of the image.
6. The system according to claim 5 , wherein the at least one processor is configured to bin sensor pixels.
7. The system according to claim 5 , wherein the at least one processor is configured to downsample an area of the image when the area of the image is characterized by a low signal to noise ratio or when the area of the image is characterized by high intensity reflectance patterns of objects in a respective area of the scene.
8. The system according to claim 1 , wherein the at least one processor is configured to use a coarse feature type from the plurality of coarse types to obtain a coarse depth estimation, and the at least one processor is configured to compute a fine depth estimation using the coarse depth estimation and a location of a fine feature type from the plurality of fine feature types in the image.
9. The system according to claim 1 , wherein the at least one processor is configured to use the coarse feature type to identify the region of interest in the image, and the at least one processor is configured to use a fine feature type within the region of interest to obtain a depth estimation for a corresponding point within the scene.
10. A computer-implemented computer implemented depth sensing method, comprising:
projecting a bi-dimensional structured light pattern onto a scene, wherein the bidimensional structured light pattern comprises a plurality of fine feature types, each one of the plurality of fine feature types is formed by a unique combination of a plurality of fine feature elements;
using a digital memory device for storing data with respect to the plurality of fine feature types, and further storing data with respect to a plurality of coarse feature types, wherein each coarse feature type from the plurality of coarse feature types is comprised of a unique combination of a plurality of coarse feature elements, wherein an appearance of a coarse feature element corresponds to a fusing of two or more fine feature elements;
capturing an image of a reflected portion of the projected bi-dimensional structured light pattern; and
using at least one processor coupled to the digital memory device for:
decoding the image of the reflected portion of the projected bi-dimensional structured light pattern and thereby assigning depth values to points in the scene;
detecting in a region of interest of the image a fine feature type from the plurality of fine feature types; and
detecting a coarse feature type from the plurality of coarse feature types.
11. The method according to claim 10 , wherein the plurality of coarse feature elements do not exist in the projected bi-dimensional structured light pattern.
12. The method according to claim 10 , wherein the plurality of coarse feature elements in the image result from projection or imaging conditions which cause respective fine feature elements to appear fused with one another in the image.
13. The method according to claim 12 , wherein the projection or imaging conditions include at least one or more of: object translucency, defocus blurring, or three-dimensional texture.
14. The method according to claim 10 , further comprising using the at least one processor for downsampling an area of the image, and estimating depth values within an area of the scene which corresponds to the downsampled area of the image using coarse feature types.
15. The method according to claim 14 , wherein the downsampling comprises binning sensor pixels.
16. The method according to claim 14 , wherein downsampling comprises downsampling the area of the image when the area of the image is characterized by a low signal to noise ratio or when the area of the image is characterized by high intensity reflectance patterns of objects in the respective area of the scene.
17. The method according to claim 10 , wherein decoding comprises:
using a coarse feature type from the plurality of coarse types to obtain a coarse depth estimation; and
computing a fine depth estimation using the coarse depth estimation and a location of a fine feature type from the plurality of fine feature types in the image.
18. The method according to claim 10 , wherein using the at least one processor comprises using a coarse feature type to identify a region of interest in the image, and using a fine feature type within the region of interest to obtain a depth estimation for a corresponding point within the scene.