IP Library › Granted Patent US 9,819,855
Granted Patent B2
US 9,819,855 · App. 14/918,730 · Granted Nov 14, 2017

Balancing exposure and gain at an electronic device based on device motion and scene distance

Inventors: Joel Hesch (Mountain View, CA); James Fung (Mountain View, CA)
Assignee: Google Inc.
H04N5/23216H04N5/145H04N5/2256H04N5/2353H04N5/23254H04N5/23258H04N9/735
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,819,855
App. No.
14/918,730
Filed
Oct 21, 2015
Granted
Nov 14, 2017
Kind
B2
Art Unit
2662
USPC
348/229.1
Abstract

An electronic device balances gain and exposure at an imaging sensor of the device based on detected image capture conditions, such as motion of the electronic device, distance of a scene from the electronic device, and predicted illumination conditions for the electronic device. By balancing the gain and exposure, the quality of images captured by the imaging sensor is enhanced, which in turn provides for improved support of location-based functionality.

Claims (62)

1. A method comprising:

identifying, at an electronic device, motion of the electronic device based on imagery captured at the electronic device and based on received non-visual pose information obtained from one or more non-visual sensors;

balancing a gain associated with an image capturing device and an exposure associated with the image capturing device based on the identified motion;

generating, at the electronic device, one or more feature descriptors for one or more spatial features identified in imagery captured using the balanced gain and exposure, wherein each of the generated feature descriptors describes one or more of an orientation, gravity direction, and scale of one or more of the identified spatial features; and

comparing the one or more generated feature descriptors to one or more known feature descriptors to identify a matched known feature descriptor.

2. The method of claim 1 , wherein the one or more non-visual sensors comprise a gyroscope of the electronic device.

3. The method of claim 1 , wherein identifying motion based on imagery comprises:

identifying a first spatial feature based on a first image captured at the electronic device; and

identifying the motion based on a difference between a first position of the first spatial feature in the first image and a second position of the first spatial feature in a second image captured at the electronic device.

4. The method of claim 3 wherein identifying motion based on imagery comprises:

selecting the first feature based on a distance of the first feature from the electronic device.

5. The method of claim 1 , wherein balancing the gain and the exposure comprises:

balancing the gain and the exposure further based on a depth of a scene indicated by the imagery.

6. The method of claim 5 , further comprising:

identifying the depth of the scene based on information captured by a depth camera of the electronic device.

7. The method of claim 1 , wherein balancing the gain and the exposure comprises:

predicting an illumination based on the motion of the electronic device; and

balancing the gain and the exposure further based on the predicted illumination.

8. The method of claim 7 , wherein predicting the illumination comprises:

identifying a current pose of the electronic device based on the one or more matched known feature descriptors and based on the received non-visual pose information;

predicting a future pose of the electronic device based on the identified motion of the electronic device; and

predicting the illumination based on the predicted future pose of the electronic device.

9. The method of claim 1 , balancing the gain and the exposure comprises:

increasing the gain and reducing the exposure in response to an increase in motion of the electronic device; and

reducing the gain and increasing the exposure in response to a decrease in motion of the electronic device.

10. A method, comprising,

identifying, at an electronic device, a distance of a scene based on imagery captured at the electronic device and based on a depth sensor of the electronic device;

balancing a gain associated with an image capturing device of the electronic device and an exposure associated with the image capturing device based on the identified distance;

generating, at the electronic device, one or more feature descriptors for one or more spatial features identified in imagery captured using the balanced gain and exposure, wherein each of the generated feature descriptors describes one or more of an orientation, gravity direction, and scale of one or more of the identified spatial features; and

comparing the one or more generated feature descriptors to one or more known feature descriptors to identify a matched known feature descriptor.

11. The method of claim 10 , wherein identifying the distance based on imagery comprises:

identifying a first feature based on a first image captured at the electronic device; and

identifying the distance based on a matched known feature descriptor associated with the first feature.

12. The method of claim 10 , wherein balancing the gain and the exposure comprises:

balancing the gain and the exposure further based on a detected motion of the electronic device.

13. The method of claim 12 , wherein the detected motion comprises a rotational velocity of the electronic device.

14. An electronic device comprising:

an imaging sensor to capture imagery;

a gain control module to control a gain of the imaging sensor;

an exposure control module to control an exposure of the imaging sensor;

non-visual sensors to capture non-visual pose information for the electronic device; and

a processor to

identify motion of the electronic device based on the imagery and based on the non-visual pose information,

balance the gain and the exposure based on the identified motion,

generate one or more feature descriptors for one or more spatial features identified in imager captured using the balanced gain and exposure, wherein each of the generated feature descriptors describes one or more of an orientation, gravity direction, and scale of one or more of the identified spatial features, and

compare the one or more generated feature descriptors to one or more known feature descriptors to identify a matched known feature descriptor.

15. The electronic device of claim 14 , wherein the processor is to identify motion based on imagery by:

identifying a first feature based on a first image captured at the electronic device; and

identifying the motion based on a difference between a first position of the first feature in the first image and a second position of the first feature in a second image captured at the electronic device.

16. The electronic device of claim 15 wherein the processor is to identify motion based on imagery by:

selecting the first feature based on a distance of the first feature from the electronic device.

17. The electronic device of claim 14 , wherein the processor is to balance the gain and the exposure by:

balancing the gain and the exposure further based on a depth of a scene indicated by the imagery.

18. The electronic device of claim 17 , wherein the processor is to:

identify the depth of the scene based on information captured by a depth camera of the electronic device.

19. The electronic device of claim 14 , wherein the processor is to balance the gain and the exposure by:

predicting an illumination based on the motion of the electronic device; and

balancing the gain and the exposure further based on the predicted illumination.

20. The electronic device of claim 19 , wherein the processor is to predict the illumination by:

identifying a current pose of the electronic device based on the imagery captured at the electronic device and based on received non-visual pose information;

predicting a future pose of the electronic device based on the identified motion of the electronic device; and

predicting the illumination based on the predicted future pose of the electronic device.

Assignments (2)
CHANGE OF NAME Recorded Oct 5, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044129/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 21, 2015
From: HESCH, JOEL; FUNG, JAMES
To: GOOGLE INC.
Reel/Frame 036844/0827 →
Continuity (1)
Related Publication 20170118400A1 · Apr 27, 2017