IP Library Granted Patent US 10,735,640
Granted Patent B2
US 10,735,640 · App. 15/892,120 · Granted Aug 4, 2020

Systems and methods for enhanced optical sensor devices

Inventors: Manoj Bikumandla (Union City, CA); Andrew Matthew Bardagjy (Fremont, CA); Cina Hazegh (Walnut Creek, CA)
Assignee: Facebook Technologies, LLC
H04N5/23212H04N5/2226H04N5/2254H04N5/23232
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,735,640
App. No.
15/892,120
Granted
Aug 4, 2020
Kind
B2
Abstract

The disclosed imaging device may include an optical sensor, a lens positioned to focus light from a scene onto the optical sensor, a matrix of variable-phase optical elements that are dimensioned to introduce at least two different phase delays into a wavefront of the light from the scene, and a housing that couples the matrix of variable-phase optical elements to the optical sensor. The housing may position the matrix at least substantially perpendicular to an optical axis of the optical sensor and may enable the optical sensor to capture at least two different perspectives of the scene per output image. Various other methods, systems, and computer-readable media are also disclosed.

Claims (59)

1. An imaging device comprising:

an optical sensor comprising a time-of-flight (ToF) depth sensor;

a lens positioned to focus light from a scene onto the optical sensor;

a matrix of variable-phase optical elements that are dimensioned to introduce at least two different phase delays into a wavefront of the light from the scene; and

a housing that couples the matrix of variable-phase optical elements to the optical sensor in a configuration that:

positions the matrix along a plane that is substantially perpendicular to an optical axis of the optical sensor, and

allows movement of the matrix along the plane enabling spatial oversampling causing the optical sensor to capture at least two different perspectives of the scene for processing into a single output image representative of the scene, wherein the ToF depth sensor determines depth values for the output image based on a phase delay of a reflected light signal relative to a reference signal.

2. The imaging device of claim 1 , further comprising: a processing subsystem programmed to:

receive, from the optical sensor, the two different perspectives of the scene; and

increase resolution of the optical sensor in at least one dimension when processing the two different perspectives of the scene to create the single output image.

3. The imaging device of claim 2 , wherein:

the processing subsystem is programmed to increase resolution of the optical sensor by performing a triangulation between the two different perspectives of the scene to provide a depth map of the scene as the output image; and

the depth map comprises depth dimension values for a plurality of portions of the scene.

4. The imaging device of claim 1 , further comprising a positioning system that couples at least one optical component of the imaging device to the housing, wherein:

the positioning system is operable to moveably reposition the at least one optical component from a first position to a second position along the plane that is substantially perpendicular to the optical axis of the optical sensor; and

the optical component comprises at least one of the lens or the matrix of variable-phase optical elements.

5. The imaging device of claim 4 , wherein:

the optical component comprises the matrix of variable-phase optical elements; and

movement of the optical component enables the spatial oversampling by causing the optical sensor to capture, at a first pixel, a different instantaneous field-of-view in the second position than in the first position.

6. The imaging device of claim 5 , further comprising a processing subsystem programmed to combine the different instantaneous fields-of-view to provide increased spatial resolution for the single output image representative of the scene.

7. The imaging device of claim 1 , wherein:

the two different perspectives of the scene each comprise a first resolution defined by an x-dimension and a y-dimension; and

the imaging device further comprises a processing subsystem programmed to oversample the scene by combining the at least two different perspectives of the scene into the single output image to generate an output image having a second resolution that is greater than the first resolution.

8. The imaging device of claim 1 , where the matrix of variable-phase optical elements comprises:

a first substantially planar optical element disposed between the lens and the optical sensor; and

a second substantially planar optical element disposed between the lens and the optical sensor, the first and second substantially planar optical elements being independently positionable to introduce the at least two different phase delays into the wavefront of the light from the scene.

9. The imaging device of claim 1 , wherein:

the at least two different phase delays are predetermined fractions of a wavelength of the light; and

the matrix of variable-phase optical elements directs:

a first portion of light to a first pixel of the optical sensor and introduces a first predetermined fractional phase delay of the at least two different phase delays into the first portion of light; and

directs a second portion of light to a second pixel of the optical sensor and introduces a second predetermined fractional phase delay of the at least two different phase delays into the second portion of light.

10. The imaging device of claim 9 , wherein:

the matrix of variable-phase optical elements directs a third portion of light having a third predetermined fractional phase delay to a third pixel of the optical sensor and directs a fourth portion of light having a fourth predetermined fractional phase delay to a fourth pixel of the optical sensor; and

the imaging device further comprises a processing subsystem programmed to perform quadrature demodulation to determine a depth between the optical sensor and an object in the scene based on sensor information from the first, second, third, and fourth pixels of the optical sensor.

11. The imaging device of claim 1 , further comprising a processing subsystem programmed to:

utilize a first signal from a first pixel of the optical sensor when the matrix is in a first position and a second signal from the first pixel when the matrix is in a second position to determine a phase difference between the light received by the first pixel and an emitted light signal projected into the scene; and

use the phase difference to calculate a depth value of a portion of the scene captured by the first pixel.

12. An imaging system comprising:

an optical sensor coupled to a head-mounted display;

a lens positioned to focus light from a scene onto the optical sensor;

a matrix of variable-phase optical elements that are dimensioned to introduce at least two different phase delays into a wavefront of the light from the scene; and

a positioning system that couples the matrix to a housing, wherein:

the positioning system is operable to moveably position an optical component of the imaging system along a plane that is substantially perpendicular to an optical axis of the optical sensor and lens;

the positioning system couples the optical component to the housing in a configuration that allows movement of the optical component along the plane enabling spatial oversampling causing the optical sensor to capture at least two different perspectives of the scene for processing into a single output image representative of the scene; and

the head-mounted display is programmed to use the two different perspectives of the scene to increase resolution of the output image in at least one dimension.

13. The imaging system of claim 12 , wherein the positioning system comprises at least one of a MEMS device or a voice coil motor that is actuatable to move the optical component at least substantially perpendicular to an optical axis.

14. The imaging system of claim 12 , wherein the optical component comprises the matrix of variable-phase optical elements such that the matrix is movable relative to the optical sensor.

15. A method comprising:

positioning a matrix of variable-phase optical elements, disposed between an optical sensor comprising a time-of-flight (ToF) depth sensor and a lens positioned to focus light from a scene onto the optical sensor, in a first position along a plane that is substantially perpendicular to an optical axis of the optical sensor;

capturing, with the optical sensor, a first perspective of the scene at the first position;

positioning the matrix of variable-phase optical elements in a second position along the plane;

capturing, with the optical sensor, a second perspective of the scene at the second position;

processing the first perspective of the scene and the second perspective of the scene to create an output image with a higher resolution than either of the captured first perspective of the scene and the second perspective of the scene; and

determining, by the TOF depth sensor, depth values for the output image based on a phase delay of a reflected light signal relative to a reference signal.

16. The method of claim 15 , further comprising increasing resolution of the optical sensor in at least one dimension other than along the optical axis.

17. The method of claim 15 , further comprising increasing resolution of the optical sensor in at least one dimension along the optical axis.

18. The method of claim 15 , wherein

capturing the first perspective of the scene at the first position comprises capturing a portion of light having a first phase delay with a first pixel of the optical sensor; and

capturing the second perspective of the scene at the second position comprises capturing a portion of light having a second phase delay with a second pixel of the optical sensor.

Assignments (3)
CHANGE OF NAME Recorded May 26, 2022
From: FACEBOOK TECHNOLOGIES, LLC
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 060199/0876 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 16, 2020
From: BIKUMANDLA, MANOJ; BARDAGJY, ANDREW MATTHEW; HAZEGH, CINA
To: OCULUS VR, LLC
Reel/Frame 051533/0240 →
CHANGE OF NAME Recorded Sep 20, 2018
From: OCULUS VR, LLC
To: FACEBOOK TECHNOLOGIES, LLC
Reel/Frame 047112/0836 →
Continuity (1)
Related Publication 20190246032A1 · Aug 8, 2019