Multi-baseline camera array system architectures for depth augmentation in VR/AR applications

1. A headset, comprising:

at least one display configured to display images;

a camera array comprising a plurality of outward facing cameras that capture images of a scene, where the plurality of outward facing cameras comprises:

a first subarray of cameras positioned at a first location on a right area of the headset, the first subarray of cameras including at least two cameras; and

a second subarray of cameras positioned at a second location on a left area of the headset, the second subarray of cameras including at least two cameras;

an illumination light source configured to project textured light onto the scene;

at least one processor; and

a memory containing an image processing application, where the image processing application directs the at least one processor to:

direct the illumination light source to project textured light onto the scene;

capture, using at least one camera of the first subarray of cameras and at least one camera of the second subarray of cameras, a set of images of the scene illuminated with the textured light;

generate depth information for the scene using the set of images;

render at least one image of the scene incorporating a virtual object positioned within the at least one image based upon the depth information; and

display the at least one rendered image on the at least one display.

2. The headset of claim 1 , wherein the cameras in the first and second, and third subarrays of cameras have the same imaging characteristics.

3. The headset of claim 1 , wherein the illumination light source is an infrared (IR) projector.

4. The headset of claim 1 , wherein the illumination light source is positioned at a third location between the first and second locations.

5. The headset of claim 1 , wherein at least one camera of the camera array is a color camera.

6. The headset of claim 1 , wherein the rendered at least one image of the scene comprises a video, and the position of the virtual object is anchored with respect to topological relationships in the scene in the video.

7. The headset of claim 1 , wherein the image processing application further directs the processor to:

capture at least one color image of the scene using the camera array; and

warp the depth information to a different viewpoint;

wherein the at least one rendered image comprises a synthesized image of the scene based upon the warped depth information and the at least one color image of the scene.

8. The headset of claim 1 , wherein the first subarray of cameras and the second subarray of cameras are symmetrically positioned across a central vertical axis of the headset.

9. The headset of claim 1 , wherein:

the first subarray of cameras comprises a first camera and a second camera, where the first and second cameras have different imaging characteristics;

the second subarray of cameras comprises a third camera and a fourth camera, where the third and fourth cameras have different imaging characteristics.

10. The headset of claim 9 , wherein:

the first and third cameras have the same imaging characteristics; and

the second and fourth cameras have the same imaging characteristics.

11. A method for displaying virtual objects in virtual reality, comprising:

projecting textured light onto a scene using an illumination light source;

capturing a set of images of the scene illuminated with the textured light using, at least one camera from a first subarray of cameras and at least one camera of a second subarray of cameras, where:

the first subarray of cameras is positioned on a left side of a headset and face outwards; and

the second subarray of cameras is positioned on a right side of the headset and face outwards;

generating, using a processor of the headset, depth information for the scene using the set of images;

rendering, using the processor of the headset and based upon the depth information, at least one image of the scene incorporating a virtual object positioned within the at least one; and

displaying the at least one rendered image on the at least one display of the headset.

12. The method of claim 11 , wherein the cameras in the first and second subarrays of cameras have the same imaging characteristics.

13. The method of claim 11 , wherein the illumination light source is an infrared (IR) projector.

14. The method of claim 11 , wherein the illumination light source is positioned at a third location between the first and second locations.

15. The method of claim 11 , wherein at least one camera of the camera array is a color camera.

16. The headset of claim 11 , wherein the rendered at least one image of the scene comprises a video, and the position of the virtual object is anchored with respect to topological relationships in the scene in the video.

17. The method of claim 11 , further comprising:

capturing at least one color image of the scene; and

warping the depth information to a different viewpoint;

wherein the at least one rendered image comprises a synthesized image of the scene based upon the warped depth information and the at least one color image of the scene.

18. The method of claim 11 , wherein the first subarray of cameras and the second subarray of cameras are symmetrically positioned across a central vertical axis of the headset.

19. The method of claim 18 , wherein:

the first subarray of cameras comprises a first camera and a second camera, where the first and second cameras have different imaging characteristics;

the second subarray of cameras comprises a third camera and a fourth camera, where the third and fourth cameras have different imaging characteristics.

20. The headset of claim 19 , wherein:

the first and third cameras have the same imaging characteristics; and

the second and fourth cameras have the same imaging characteristics.