IP Library Granted Patent US 9,407,837
Granted Patent B2
US 9,407,837 · App. 14/186,877 · Granted Aug 2, 2016

Depth sensor using modulated light projector and image sensor with color and IR sensing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,407,837
App. No.
14/186,877
Granted
Aug 2, 2016
Kind
B2
Abstract

An electronic device includes a modulated light projector to project an infrared light pattern and an image sensor. The image sensor includes an array of photosensors and a filter array overlying the array of photosensors. The filter array comprises a mosaic of color filter elements and infrared filter elements, each color filter element overlying a corresponding photosensor of a first subset of photosensors and each infrared filter element overlying a corresponding photosensor of a second subset of photosensors. The electronic device further includes a controller to, in a first mode, control the image sensor to capture a color image using the first subset of photosensors and, in a second mode, control the image sensor to capture a depth image representing a reflection of the infrared light pattern using the second subset of photosensors.

Claims (31)

1. An electronic device comprising:

a modulated light projector to project an infrared light pattern; and

an image sensor comprising:

an array of photosensors; and

a filter array overlying the array of photosensors and comprising a mosaic of color filter elements and infrared filter elements, each color filter element overlying a corresponding photosensor of a first subset of photosensors and each infrared filter element overlying a corresponding photosensor of a second subset of photosensors, wherein each photosensor of the first subset has a first size and each photosensor of the second subset has a second size different than the first size; and

a controller to, in a first mode, control the image sensor to capture a color image using the first subset of photosensors and, in a second mode, control the image sensor to capture a depth image representing a reflection of the infrared light pattern using the second subset of photosensors.

2. The electronic device of claim 1 , wherein the modulated light projector comprises:

an array of one or more vertical cavity surface emitting laser (VCSEL) diodes;

an array of one or more lenses overlying the array of one or more VCSEL diodes; and

a diffractive optical element overlying the array of one or more lenses.

3. The electronic device of claim 1 , wherein the electronic device further comprises:

a display disposed at a first surface of the electronic device, wherein the electronic device is configured to present, via the display, imagery captured using the image sensor; and

wherein the modulated light projector and the image sensor are disposed at a second surface of the electronic device, the second surface opposite the first surface.

4. The electronic device of claim 3 , wherein:

the image sensor is a first image sensor; and

the electronic device further comprises a second image sensor disposed at the second surface, the second image sensor having a field of view different than a field of view of the first image sensor.

5. A method comprising:

providing an electronic device comprising an image sensor, the image sensor comprising an array of photosensors and a filter array overlying the array of photosensors and comprising a mosaic of color filter elements and infrared filter elements, each color filter element overlying a corresponding photosensor of a first subset of photosensors and each infrared filter element overlying a corresponding photosensor of a second subset of photosensors;

capturing, at the electronic device, a color image using the first subset of photosensors;

projecting, from the electronic device, an infrared light pattern;

capturing, at the electronic device, a depth image representing a reflection of the infrared light pattern using the second subset of photosensors; and

wherein providing the electronic device comprises providing the image sensor with each photosensor of the first subset having a first size and each photosensor of the second subset having a second size different than the first size.

6. The method of claim 5 , further comprising:

determining, at the electronic device, at least one spatial feature from at least one of the color image and the depth image; and

determining, at the electronic device, at least one of a relative position and a relative orientation of the electronic device based on the at least one spatial feature.

7. The method of claim 5 , further comprising:

displaying the color image at a display of the electronic device.

8. The method of claim 5 , wherein capturing the depth image comprises:

binning subarrays of M×N photosensors of the second subset into corresponding macropixels for the depth image, M and N comprising integer numbers greater than zero and at least one of M and N being greater than one.

9. The method of claim 5 , wherein providing the image sensor further comprises:

providing the image sensor with the color filter elements that are non-uniformly distributed in the filter array.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044566/0657 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 20, 2015
From: LEE, JOHNNY C.
To: GOOGLE INC.
Reel/Frame 035213/0612 →