Multi-baseline camera array system architectures for depth augmentation in VR/AR applications
Embodiments of the invention provide a camera array imaging architecture that computes depth maps for objects within a scene captured by the cameras, and use a near-field sub-array of cameras to compute depth to near-field objects and a far-field sub-array of cameras to compute depth to far-field objects. In particular, a baseline distance between cameras in the near-field subarray is less than a baseline distance between cameras in the far-field sub-array in order to increase the accuracy of the depth map. Some embodiments provide an illumination near-IR light source for use in computing depth maps.
1. A headset comprising: a wearable frame with a left side and a right side; and a camera array, wherein the camera array includes:
a first subarray of cameras positioned on the left side of the frame and includes at least two cameras configured to capture images of a first scene or video of a first scene, wherein each camera in the first subarray possesses the same orientation relative to each other camera in the first subarray;
a second subarray of cameras positioned on the right side of the frame and includes at least two cameras configured to capture images of a second scene or video of a second scene, wherein each camera in the second subarray possesses the same orientation relative to each other camera in the second subarray, and wherein a baseline distance between the first and second subarrays is 12-14 centimeters (cm) and the baseline distance is greater than a baseline distance between each of the cameras in the first subarray and each of the cameras in the second subarray;
a third subarray of cameras positioned between the first subarray and the second subarray and includes at least one reference camera configured to capture images of a third scene or video of a third scene; and
an illumination light source positioned within a threshold distance from the at least one reference camera.
2. The headset of claim 1 , wherein the cameras in the first, second, and third subarrays of cameras have the same imaging characteristics.
3. The headset of claim 1 , wherein the cameras in the first and second subarrays of cameras have different imaging characteristics from the at least one reference camera.
4. The headset of claim 1 , wherein the cameras in the first and second subarrays of cameras each have a different epipolar axis with the reference camera.
5. The headset of claim 1 , wherein the cameras in the first and second subarrays of cameras are all monochrome cameras with a same field of view.
6. The headset of claim 5 , wherein the field of view is selected from the group consisting of 70 degrees and 90 degrees.
7. The headset of claim 1 , wherein the first, second, and third scenes are the same scene.
8. The headset of claim 1 , wherein:
the cameras in the first subarray are configured to generate a depth map of the first scene using parallax shift of an object in the first scene between images obtained from each camera in the first subarray; or
the cameras in the second subarray are configured to generate a depth map of the second scene using parallax shift of an object in the second scene between images obtained from each camera in the second subarray.
9. The headset of claim 8 , wherein the depth map of the first scene is generated by performing a disparity search that detects a shift of an object in each camera in the first subarray to calculate the distance to the object based upon a baseline distance between each camera in the first subarray and the focal lengths of each camera in the first subarray; and
the depth map of the second scene is generated by performing a disparity search that detects a shift of an object in each camera in the second subarray to calculate the distance to the object based upon a baseline distance between each camera in the second subarray and the focal lengths of each camera in the second subarray.
10. The headset of claim 1 , wherein each camera in the first and second subarrays possess the same orientation as each other camera in the first and second subarrays.