IP Library Granted Patent US 10,440,349
Granted Patent B2
US 10,440,349 · App. 15/717,332 · Granted Oct 8, 2019

3-D 360 degrees depth projector

Inventors: Joseph Duggan (San Francisco, CA); Cina Hazegh (Walnut Creek, CA); Fei Liu (San Jose, CA)
Assignee: Facebook Technologies, LLC
H04N13/271H04N5/23238H04N13/254H04N13/282H04N13/296H04N5/247
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,440,349
App. No.
15/717,332
Granted
Oct 8, 2019
Kind
B2
Abstract

A camera system configured to generate depth information for a local area. The camera system comprises a plurality of depth camera sub-assemblies arranged in a substantially spherical arrangement. Each sub-assembly comprises a projector that projects a structured light pattern onto a portion of the local area, such that the projected light patterns of the plurality of sub-assemblies form a tiled light pattern covering 360 degrees of the local area. Each sub-assembly further comprises at least one camera is configured to capture images of the local area. A controller of the camera system is configured to receive the captured images and to construct a 360 degree depth map of the scene, based upon the structured light patterns projected by the projectors of the plurality captured in the received images.

Claims (37)

1. A camera system comprising:

a plurality of depth camera sub-assemblies arranged in a substantially spherical arrangement around a center point, the plurality of depth camera sub-assemblies together having a 360 degree view of a local area surrounding the camera system, and each depth camera sub-assembly comprising:

a projector facing away from the center point and configured to project a structured light pattern over a field of view of the local area, the projected structured light pattern being part of a tiled structured light pattern that is projected into 360 degrees of the local area formed by respective structured light patterns projected by projectors of the plurality of depth camera sub-assemblies, wherein the respective structured light patterns are time-multiplexed such that each projector of at least a portion of the plurality of depth camera sub-assemblies projects its respective structured light pattern over a different time period;

a plurality of cameras, each positioned facing away from the center point, and configured capture one or more images of at least a portion of the structured light pattern projected by the projector, wherein the projector of the depth camera sub-assembly is surrounded by the plurality of cameras; and

a controller configured to:

receive the one or more images from the at least one camera of each of the plurality of depth camera sub-assemblies;

construct a 360 degree depth map of the local area, based upon the structured light patterns projected by the projectors of the plurality of depth camera sub-assemblies in the received images.

2. The camera system of claim 1 , wherein the projector of each of the plurality of depth camera sub-assemblies is positioned such that an optical axis of the projector intersects the center point.

3. The camera system of claim 1 , wherein the at least one camera of each of the plurality of depth camera sub-assemblies is positioned such that an optical axis of the at least one camera intersects the center point.

4. The camera system of claim 1 , wherein a depth camera sub-assembly, of the plurality of depth camera sub-assemblies, comprises a plurality of cameras having overlapping fields of view.

5. The camera system of claim 1 , wherein the projectors of the plurality of depth camera sub-assemblies are configured to project respective structured light patterns associated with different frequency ranges.

6. The camera system of claim 1 , wherein a portion of the local area is within a field of view of at least two cameras of the at least one camera of each of the plurality of depth camera sub-assemblies, and wherein the controller is configured to construct a portion of the 360 degree depth map corresponding to the portion of the local area based upon stereo images of the local area captured by the at least two cameras.

7. A camera system comprising:

a plurality of depth camera sub-assemblies arranged in a substantially spherical arrangement around a center point, the plurality of depth sub-camera assemblies together having a 360 degree view of a local area surrounding the camera system, each depth camera sub-assembly comprising:

a projector facing away from the center point and configured to project a structured light pattern over a field of view of the local area, the projected structured light pattern being part of a tiled structured light pattern that is projected into 360 degrees of the local area formed by respective structured light patterns projected by projectors of the plurality of depth camera sub-assemblies, wherein the respective structured light patterns are time-multiplexed such that each projector of at least a portion of the plurality of depth camera sub-assemblies projects its respective structured light pattern over a different time period;

a plurality of cameras, each positioned facing away from the center point, and configured to capture one or more images of the local area in accordance with the imaging instructions, wherein the projector of the depth camera sub-assembly is surrounded by the plurality of cameras; and

at least one controller configured to:

provide imaging instructions to the plurality of depth camera sub-assemblies;

receive the one or more images from the at least one camera of each of the plurality of depth camera sub-assemblies;

construct a 360 degree depth map of the local area, based in part on the structured light patterns projected by the projectors of the plurality of depth camera sub-assemblies in one or more of the received images.

8. The camera system of claim 7 , wherein the projector of each of the plurality of depth camera sub-assemblies is positioned such that an optical axis of the projector intersects the center point, and wherein the at least one camera of each of the plurality of depth camera sub-assemblies is positioned such that an optical axis of the at least one camera intersects the center point.

9. The camera system of claim 7 , wherein a depth camera sub-assembly, of the plurality of depth camera sub-assemblies, comprises a plurality of cameras having overlapping fields of view.

10. The camera system of claim 7 , wherein the imaging instructions are configured to cause the projectors of the plurality of depth camera sub-assemblies to project respective structured light patterns associated with different frequency ranges.

11. The camera system of claim 7 , wherein a portion of the local area is within a field of view of at least two cameras of the at least one camera of each of the plurality of depth camera sub-assemblies, and wherein the controller is configured to construct a portion of the 360 degree depth map corresponding to the portion of the local area based upon stereo images of the local area captured by the at least two cameras.

12. A camera system comprising:

a plurality of depth camera sub-assemblies arranged in a substantially spherical arrangement around a center point, the plurality of depth camera sub-assemblies together having a 360 degree view of a local area surrounding the camera system, each depth camera sub-assembly comprising:

a projector facing away from the center point and configured to project a structured light pattern over a field of view of the local area, the projected structured light pattern being part of a tiled structured light pattern that is projected into 360 degrees of the local area formed by respective structured light patterns projected by projectors of the plurality of depth camera sub-assemblies, wherein the respective structured light patterns are time-multiplexed such that each projector of at least a portion of the plurality of depth camera sub-assemblies projects its respective structured light pattern over a different time period;

a plurality of cameras, each positioned facing away from the center point, and configured to capture one or more images of the local area in accordance with the imaging instructions, wherein the projector of the depth camera sub-assembly is surrounded by the plurality of cameras; and

at least one controller configured to:

provide imaging instructions to the plurality of depth camera sub-assemblies;

receive the one or more images from the at least one camera of each of the plurality of depth camera sub-assemblies;

construct a 360 degree depth map of the local area, based upon the structured light patterns projected by the projectors of the plurality of structured light sub-assemblies in the received images; and

provide some of the images and the 360 degree depth map to a processing server, wherein the processing server generates 360 degree-3 dimensional (3D) content using the provided images and the 360 degree depth map.

13. The camera system of claim 12 , wherein the projector of each of the plurality of depth camera sub-assemblies is positioned such that an optical axis of the projector intersects the center point, and wherein the at least one camera of each of the plurality of depth camera sub-assemblies is positioned such that an optical axis of the at least one camera intersects the center point.

14. The camera system of claim 12 , wherein a depth camera sub-assembly, of the plurality of depth camera sub-assemblies, comprises a plurality of cameras having overlapping fields of view.

15. The camera system of claim 12 , wherein the imaging instructions are configured to cause the projectors of the plurality of depth camera sub-assemblies to project respective structured light patterns associated with different frequency ranges.

16. The camera system of claim 12 , wherein a portion of the local area is within a field of view of at least two cameras of the at least one camera of each of the plurality of depth camera sub-assemblies, and wherein the controller is configured to construct a portion of the 360 degree depth map corresponding to the portion of the local are based upon stereo images of the local area captured by the at least two cameras.

Assignments (3)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060315/0224 →
CHANGE OF NAME Recorded Sep 12, 2018
From: OCULUS VR, LLC
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 047178/0616 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 15, 2017
From: DUGGAN, JOSEPH; HAZEGH, CINA; LIU, FEI
To: OCULUS VR, LLC
Reel/Frame 044407/0207 →
Continuity (1)
Related Publication 20190098276A1 · Mar 28, 2019