Multi-baseline camera array system architectures for depth augmentation in VR/AR applications
Embodiments of the invention provide a camera array imaging architecture that computes depth maps for objects within a scene captured by the cameras, and use a near-field sub-array of cameras to compute depth to near-field objects and a far-field sub-array of cameras to compute depth to far-field objects. In particular, a baseline distance between cameras in the near-field subarray is less than a baseline distance between cameras in the far-field sub-array in order to increase the accuracy of the depth map. Some embodiments provide an illumination near-IR light source for use in computing depth maps.
1. A method of estimating distances to objects within a scene from a set of images captured from different cameras in a set of cameras using a processor configured by an image processing application, the method comprising:
generating an initial depth map of a scene;
identify near-field and far-field portions of the scene;
refine the depth map for the near-field portions of the scene using image data captured from a near-field set of cameras; and
refine the depth map for the far-filed portions of the scene using image data captured from a far-field set of cameras, wherein a baseline distance between cameras in the near-field set of cameras is less than a baseline distance between cameras in the far-field set of cameras.
2. The method of claim 1 , further comprising illuminating the scene using an illumination light source positioned within a threshold distance from a reference camera.
3. The method of claim 1 , wherein the near-field portions of the scene are at a distance less than a certain threshold and the far-field portions of the scene are at a distance greater than the certain threshold.
4. The method of claim 1 , wherein each camera in the set of cameras has the same imaging properties.
5. The method of claim 1 , where the set of cameras have different field of views.
6. The method of claim 1 , wherein the set of cameras have different resolutions.
7. The method of claim 1 , wherein the set of cameras image in different portions of a light spectrum.
8. The method of claim 1 , wherein the set of cameras are monochrome cameras with a same field of view and resolution.
9. The method of claim 8 , wherein the monochrome cameras image across at least a portion of the visible spectrum and at least a portion of the near-infrared (near-IR) spectrum.
10. The method of claim 1 , further comprising using the depth map in order to determine where to render an object on a display of a virtual reality (VR) application such that the object is appropriately occluded by a real world object also being displayed within the VR application.
11. The method of claim 1 , further comprising using the depth map in order to determine where to render an object on a display of an augmented reality (AR) application such that the object is appropriately occluded by a real world object visible through a transparent display within the AR application.
12. The method of claim 1 , wherein the set of cameras are mounted within a headset that includes a display via which images can be displayed.
13. The method of claim 12 , wherein a reference camera and the display are part of a removable component that is mounted within the headset.
14. A non-transitory machine readable medium containing processor instructions for estimating distances to objects within a scene from a set of images captured from different cameras in a set of cameras, where execution of the instructions by a processor causes the processor to perform a process that comprises:
generating an initial depth map of a scene;
identify near-field and far-field portions of the scene;
refine the depth map for the near-field portions of the scene using image data captured from a near-field set of cameras; and
refine the depth map for the far-filed portions of the scene using image data captured from a far-field set of cameras, wherein a baseline distance between cameras in the near-field set of cameras is less than a baseline distance between cameras in the far-field set of cameras.
15. The non-transitory machine readable medium of claim 14 , further comprising illuminating the scene using an illumination light source positioned within a threshold distance from a reference camera.
16. T The non-transitory machine readable medium of claim 14 , wherein the near-field portions of the scene are at a distance less than a certain threshold and the far-field portions of the scene are at a distance greater than the certain threshold.
17. The non-transitory machine readable medium of claim 14 , wherein each camera in the set of cameras has the same imaging properties.
18. The non-transitory machine readable medium of claim 14 , where the set of cameras have different field of views.
19. The non-transitory machine readable medium of claim 14 , wherein the set of cameras have different resolutions.