IP Library Granted Patent US 9,674,460
Granted Patent B1
US 9,674,460 · App. 14/945,893 · Granted Jun 6, 2017

Generating high-dynamic range images using varying exposures

Inventors: Mark Allen Robertson (Cupertino, CA); Ken James Kryda (Sunnyvale, CA)
Assignee: Google Inc.
H04N5/2355H04N5/2353
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,674,460
App. No.
14/945,893
Granted
Jun 6, 2017
Kind
B1
Abstract

Systems and methods are provided for generating high-dynamic range images. In particular, a signal-to-noise (SNR) ratio objective associated with an imaging platform configured to capture one or more images of a region of interest can be determined. The SNR objective can specify a desired signal-to-noise ratio behavior as a function of a brightness of the region of interest. An exposure profile associated with the imaging platform can then be determined based at least in part on the SNR objective. Data indicative of a plurality of image frames, each depicting at least a portion of the region of interest, can then be obtained. The plurality of image frames being captured by the imaging platform in accordance with the exposure profile.

Claims (30)

1. A computer-implemented method of creating high-dynamic range images, the method comprising:

determining, by one or more computing devices, a signal-to-noise ratio objective associated with an imaging platform configured to capture one or more images region of interest, the signal-to-noise ratio objective specifying a desired signal-to-noise behavior as a function of a brightness of the region of interest,

wherein the signal-to-noise ratio objective is to shape a signal-to-noise ratio curve based at least in part on a brightness distribution, the one or more computing devices determining the brightness distribution based at least in part on a location of the imaging platform, a location of the sun relative to the earth, an albedo associated with the region of interest, atmospheric absorption, an expected reflectance of one or more targets within the region of interest, or expected atmospheric conditions;

determining, by the one or more computing devices, an exposure profile associated with the imaging platform based at least in part on the signal-to-noise ratio objective;

obtaining, by the one or more computing devices, data indicative of a plurality of image frames, each depicting at least a portion of the region of interest, the plurality of image frames being captured by the imaging platform in accordance with the exposure profile; and

generating, by the one or more computing devices, a high-dynamic range of at least a portion of the region of interest based at least in part on the plurality of image frames.

2. The computer-implemented method of claim 1 , wherein the exposure profile comprises a plurality of integrations times to be used in capturing the plurality of image frames.

3. The computer-implemented method of claim 2 , wherein the exposure profile specifies an integration time for each image frame to be captured.

4. The computer-implemented method of claim 2 , wherein the plurality of integration times are determined to shape a signal-to-noise ratio curve in accordance with the signal-to-noise ratio objective.

5. The computer-implemented method of claim 2 , wherein each image frame of the plurality of image frames is captured using a different integration time from the exposure profile.

6. The computer-implemented method of claim 1 , wherein capturing, by the one or more computing devices, a plurality of image frames depicting the region of interest comprises scanning the region of interest using a two-dimensional image sensor.

7. The computer-implemented method of claim 5 , wherein the two-dimensional image sensor is a two-dimensional staring sensor configured to capture a two-dimensional image frame in a single snapshot.

8. The computer-implemented method of claim 6 , wherein the two-dimensional staring sensor comprises a complementary metal-oxide-semiconductor sensor, a charge coupled device sensor, or an active-pixel sensor.

9. The computer-implemented method of claim 1 , wherein the signal-to-noise ratio objective is to maintain a signal-to-noise ratio curve within a predetermined threshold range.

10. The computer-implemented method of claim 1 , wherein the brightness distribution is determined prior to capturing the plurality of image frames.

11. The computer-implemented method of claim 1 , wherein determining, by the one or more computing devices, a brightness distribution associated with the region of interest comprises determining a brightness distribution associated with at least one image frame of the plurality of image frames.

12. A computing system, comprising: one or more processors; and

one or more memory devices, the one or more memory devices storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising:

determining a signal-to-noise ratio objective associated with an imaging platform configured to capture one or more images of a region of interest, the signal-to-noise ratio objective specifying a desired signal-to-noise curve as a function of a brightness of the region of interest,

wherein the signal-to-noise ratio objective is to shape the signal-to-noise ratio curve based at least in part on a brightness distribution, the one or more computing devices determining the brightness distribution based at least in part on a location of the imaging platform, a location of the sun relative to the earth, an expected reflectance of one or more targets within the region of interest, or expected atmospheric conditions;

determining, by the one or more computing devices, an exposure profile associated with the imaging platform based at least in part on the signal-to-noise ratio objective;

obtaining, by the one or more computing devices, data indicative of a plurality of image frames, each depicting at least a portion of the region of interest, the plurality of image frames being captured by the imaging platform in accordance with the exposure profile; and

generating, by the one or more computing devices, a high-dynamic range of at least a portion of the region of interest based at least in part on the plurality of image frames.

13. The computing system of claim 12 , wherein the exposure profile comprises a plurality of integrations times to be used in capturing the plurality of image frames.

14. One or more tangible, non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

determining a signal-to-noise ratio objective associated with an imaging platform configured to capture one or more images of a region of interest, the signal-to-noise ratio objective specifying a desired signal-to-noise curve as a function of a brightness of the region of interest,

wherein the signal-to-noise ratio objective is to shape a signal-to-noise ratio curve based at least in part on a brightness distribution, the one or more computing devices determining the brightness distribution based at least in part on a location of the imaging platform, a location of the sun relative to the earth, an expected reflectance of one or more targets within the region of interest, or expected atmospheric conditions;

determining, by the one or more computing devices, an exposure profile associated with the imaging platform based at least in part on the signal-to-noise ratio objective;

obtaining, by the one or more computing devices, data indicative of a plurality of image frames, each depicting at least a portion of the region of interest, the plurality of image frames being captured by the imaging platform in accordance with the exposure profile; and

generating, by the one or more computing devices, a high-dynamic range of at least a portion of the region of interest based at least in part on the plurality of image frames.

Assignments (8)
MERGER AND CHANGE OF NAME Recorded May 5, 2022
From: PLANET LABS INC.; PLANET LABS PBC
To: PLANET LABS PBC
Reel/Frame 059857/0587 →
RELEASE OF SECURITY INTEREST Recorded Dec 10, 2021
From: SILICON VALLEY BANK
To: PLANET LABS INC.; TERRA BELLA TECHNOLOGIES INC.; PL FOREIGN HOLDCO, INC.
Reel/Frame 058359/0501 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jun 21, 2019
From: PLANET LABS INC.; PL INTERMEDIATE TB, INC.; PLANET LABS TB, INC.; TERRA BELLA TECHNOLOGIES INC.; PLANET LABS LLC; PL FOREIGN HOLDCO, INC.
To: SILICON VALLEY BANK
Reel/Frame 049558/0515 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 30, 2017
From: PLANET LABS TB, INC.
To: PLANET LABS, INC.
Reel/Frame 044260/0159 →
CORRECTIVE ASSIGNMENT TO CORRECT THE INCORRECT SERIAL NO. 15/061,851 PREVIOUSLY RECORDED AT REEL: 043277 FRAME: 0669. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 18, 2017
From: GOOGLE INC.
To: PLANET LABS TB, INC.
Reel/Frame 043661/0060 →
CORRECTIVE ASSIGNMENT TO CORRECT THE NAME OF THE ASSIGNEE PREVIOUSLY RECORDED AT REEL: 043277 FRAME: 0669. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 17, 2017
From: GOOGLE INC.
To: PLANET LABS TB, INC.
Reel/Frame 043409/0837 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 14, 2017
From: GOOGLE INC.
To: PLANT LABS TB, INC.
Reel/Frame 043277/0669 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 19, 2015
From: ROBERTSON, MARK ALLEN; KRYDA, KEN JAMES
To: GOOGLE INC.
Reel/Frame 037088/0985 →