IP Library › Granted Patent US 10,388,026
Granted Patent B1
US 10,388,026 · App. 15/643,912 · Granted Aug 20, 2019

Fast scanning large field-of-view devices for depth sensing

Inventors: Qing Chao (Redmond, WA); Lu Lu (Seattle, WA)
Assignee: Facebook Technologies, LLC
G06T7/521G02B27/281G02F1/292G02F1/33G06T7/557H04N5/2256H04N5/2354G02F2201/305G06F3/013G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,388,026
App. No.
15/643,912
Granted
Aug 20, 2019
Kind
B1
Abstract

A depth camera assembly (DCA) for depth sensing of a local area. The DCA includes a structured light generator, an imaging device, and a controller. The structured light generator illuminates the local area with structured light. The structured light generator includes an acousto-optic deflector (AOD) and a liquid crystal device (LCD). The AOD functions as one or more dynamic diffraction gratings that diffract one or more optical beams to form diffracted scanning beams. The LCD includes one or more liquid crystal gratings (LCGs) that diffract the diffracted scanning beams to form the structured light projected into the local area. The imaging device captures portions of the structured light reflected from one or more objects in the local area. The controller determines depth information for the one or more objects based on the captured portions of the reflected structured light.

Claims (69)

1. A depth camera assembly (DCA) comprising:

a structured light generator configured to illuminate a local area with structured light in accordance with emission instructions, the structured light generator comprising:

an illumination source configured to emit one or more optical beams,

an acousto-optic deflector (AOD) that generates, based in part on the emission instructions, diffracted scanning beams from the one or more optical beams, the AOD functions as one or more dynamic diffraction gratings that diffract the one or more optical beams to form the diffracted scanning beams,

a liquid crystal device (LCD) that generates, based in part on the emission instructions, the structured light from the diffracted scanning beams, the LCD includes one or more liquid crystal gratings (LCGs) that diffract the diffracted scanning beams, and

a projection assembly configured to project the structured light into the local area;

an imaging device configured to capture portions of the structured light reflected from one or more objects in the local area; and

a controller configured to:

generate the emission instructions,

provide the emission instructions to the structured light generator, and

determine depth information for the one or more objects based at least in part on the captured portions of the reflected structured light.

2. The DCA of claim 1 , wherein the AOD includes a plurality of acousto-optic devices configured to function as a plurality of dynamic diffraction gratings diffracting the one or more optical beams incident to the plurality of dynamic diffraction gratings at an angle that satisfies the Bragg matching condition to form the diffracted scanning beams based in part on the emission instructions.

3. The DCA of claim 1 , wherein:

the AOD includes a single acousto-optic device;

the LCD includes a single LCG of the one or more LCGs coupled to the single acousto-optic device; and

the single LCG is configured to expand scanning angles of the diffracted scanning beams along a first dimension and along a second dimension orthogonal to the first dimension to generate the structured light.

4. The DCA of claim 1 , wherein the AOD includes at least one pair of acousto-optic devices configured to function as at least one pair of dynamic diffraction gratings whose axes of orientation are orthogonal to each other, the at least one pair of acousto-optic devices generating the diffracted scanning beams as two-dimensional scanning beams.

5. The DCA of claim 4 , wherein:

the controller is further configured to:

create the emission instructions which include at least one first radio frequency at which a first acousto-optic device of the at least one pair of acousto-optic devices is driven and at least one second radio frequency at which a second acousto-optic device of the at least one pair of acousto-optic devices is driven, and

modify the at least one first radio frequency and the at least one second radio frequency to adjust a diffraction angle of the first acousto-optic device and a diffraction angle of the second acousto-optic device to form the diffracted scanning beams,

wherein the first acousto-optic device includes at least one first transducer and at least one first diffraction area, and responsive to the at least one first radio frequency in the emission instructions, the at least one first transducer is configured to generate at least one first sound wave in the at least one first diffraction area propagating in a first direction to form a first dynamic diffraction grating of the one or more dynamic diffraction gratings, and

the second acousto-optic device includes at least one second transducer and at least one second diffraction area, and responsive to the at least one second radio frequency in the emission instructions, the at least one second transducer is configured to generate at least one second sound wave in the at least one second diffraction area propagating in a second direction orthogonal to the first direction to form a second dynamic diffraction grating of the one or more dynamic diffraction gratings.

6. The DCA of claim 4 , wherein:

the LCD includes a first LCG and a second LCG of the one or more LCGs coupled to the at least one pair of acousto-optic devices;

the first LCG is configured to expand scanning angles of the diffracted scanning beams along a first dimension; and

the second LCG is configured to expand scanning angles of the diffracted scanning beams along a second dimension orthogonal to the first dimension to generate the structured light.

7. The DCA of claim 4 , wherein:

the LCD includes a single LCG of the one or more LCGs coupled to the at least one pair of acousto-optic devices; and

the single LCG is configured to expand scanning angles of the diffracted scanning beams along a first dimension and along a second dimension orthogonal to the first dimension to generate the structured light.

8. The DCA of claim 1 , wherein the AOD includes one or more acousto-optic devices, each acousto-optic device in the AOD implemented as a thin film acousto-optic device.

9. The DCA of claim 1 , wherein the AOD includes one or more acousto-optic devices, each acousto-optic device in the AOD implemented as a plate acousto-optic device.

10. The DCA of claim 1 , wherein the AOD includes one or more acousto-optic devices, each acousto-optic device in the AOD implemented as a bulk acousto-optic device.

11. The DCA of claim 1 , wherein the one or more optical beams are incident on the AOD at an angle that satisfies the Bragg matching condition.

12. The DCA of claim 1 , wherein the one or more optical beams are temporally modulated providing the structured light to be temporally modulated.

13. The DCA of claim 1 , wherein the controller is further configured to determine the depth information based on at least one of: information about a pattern of the reflected structured light distorted by shapes of the one or more objects, and time-of-flight information associated with the reflected structured light.

14. The DCA of claim 1 , wherein the imaging device includes:

a camera; and

a polarizing element positioned in front of the camera, the polarizing element configured to propagate the captured portions of the reflected structured light having a polarization orthogonal to a polarization of the structured light, and

the controller is further configured to determine the depth information for the one or more objects based in part on the polarization of the reflected structured light captured by the camera.

15. A method comprising:

generating emission instructions;

generating, based in part on the emission instructions, diffracted scanning beams from one or more optical beams by diffracting the one or more optical beam using an acousto-optic deflector (AOD) that functions as one or more dynamic diffraction gratings to form the diffracted scanning beams;

generating, based in part on the emission instructions, structured light from the diffracted scanning beams by diffracting the diffracted scanning beams using a liquid crystal device (LCD) comprising one or more liquid crystal gratings (LCGs);

projecting the structured light into a local area;

capturing portions of the structured light reflected from one or more objects in the local area; and

determining depth information for the one or more objects based at least in part on the captured portions of the reflected structured light.

16. The method of claim 15 , wherein generating the diffracted scanning beams comprises:

diffracting the one or more optical beams using a single acousto-optic device of the AOD configured to function as a dynamic diffraction grating to form the diffracted scanning beams based in part on the emission instructions.

17. The method of claim 15 , wherein generating the structured light comprises:

expanding, using a single LCG of the one or more LCGs in the LCD, scanning angles of the diffracted scanning beams along a first dimension and along a second dimension orthogonal to the first dimension to form the structured light.

18. The method of claim 15 , wherein generating the diffracted scanning beams comprises:

diffracting the one or more optical beams using at least one pair of acousto-optic devices of the AOD configured to function as at least one pair of dynamic diffraction gratings whose axes of orientation are orthogonal to each other to form the diffracted scanning beams as two-dimensional scanning beams.

19. The method of claim 15 , wherein generating the structured light comprises:

expanding, using a first LCG of the one or more LCGs in the LCD, scanning angles of the diffracted scanning beams along a first dimension; and

expanding, using a second LCG of the one or more LCGs in the LCD, scanning angles of the diffracted scanning beams along a second dimension orthogonal to the first dimension to form the structured light.

20. A head-mounted display (HMD) comprising:

a display configured to emit image light;

a structured light generator configured to illuminate a local area with structured light in accordance with emission instructions, the structured light generator comprising:

an illumination source configured to emit one or more optical beams,

an acousto-optic deflector (AOD) that generates, based in part on the emission instructions, diffracted scanning beams from the one or more optical beams, the AOD functions as one or more dynamic diffraction gratings that diffract the one or more optical beams to form the diffracted scanning beams,

a liquid crystal device (LCD) that generates, based in part on the emission instructions, the structured light from the diffracted scanning beams, the LCD includes one or more liquid crystal gratings (CLGs) that diffract the diffracted scanning beams, and

a projection assembly configured to project the structured light into the local area;

an imaging device configured to capture portions of the structured light reflected from one or more objects in the local area;

a controller configured to:

generate the emission instructions,

provide the emission instructions to the structured light generator, and

determine depth information for the one or more objects based at least in part on the captured portions of the reflected structured light; and

an optical assembly configured to direct the image light to an exit pupil of the HMD corresponding to a location of a user's eye, the image light comprising the determined depth information.

Assignments (3)
CHANGE OF NAME Recorded Jun 8, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060315/0224 →
CHANGE OF NAME Recorded Sep 12, 2018
From: OCULUS VR, LLC
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 047178/0616 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 20, 2017
From: CHAO, QING; LU, LU
To: OCULUS VR, LLC
Reel/Frame 043054/0211 →
Cited By (2)
US 12,250,442 US 12,722,226