IP Library Granted Patent US 9,087,391
Granted Patent B2
US 9,087,391 · App. 13/713,734 · Granted Jul 21, 2015

Determining an image capture payload burst structure

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,087,391
App. No.
13/713,734
Granted
Jul 21, 2015
Kind
B2
Abstract

A first plurality of images of a scene may be captured. Each image of the first plurality of images may be captured using a different TET. Based at least on the first plurality of images, a long TET, a short TET, and a TET sequence that includes the long TET and the short TET may be determined. A second plurality of images of the scene may be captured. The images in the second plurality of images may be captured sequentially in an image sequence using a sequence of TETs corresponding to the TET sequence. Based on one or more images in the image sequence, an output image may be constructed.

Claims (74)

1. A method comprising:

capturing, by an image sensor, a first plurality of images of a scene, wherein each image of the first plurality of images is captured using a different total exposure time (TET);

based at least on the first plurality of images, determining a long TET, a short TET, and a TET sequence that includes the long TET and the short TET;

capturing, by the image sensor, a second plurality of images of the scene, wherein the images in the second plurality of images are captured sequentially in an image sequence using a sequence of TETs corresponding to the TET sequence, wherein the image sequence includes a three-image sub-sequence of a first long-TET image captured using the long TET, followed by a short-TET image captured using the short TET, followed by a second long-TET image captured using the long TET; and

based on one or more images in the image sequence, constructing an output image.

2. The method of claim 1 , wherein the image sequence also includes a second three-image sub-sequence of a first third long-TET image captured using the long TET, followed by a second fourth long-TET image captured using the long TET, followed by a second short-TET image captured using the short TET.

3. The method of claim 1 , wherein the image sequence includes a two-image sub-sequence of a primary short-TET image followed by a primary long-TET image, wherein the image sequence also includes one or more secondary short-TET images and one or more secondary long-TET images, wherein the primary short-TET image and the secondary short-TET images are captured using the short TET, wherein the primary long-TET image and the secondary long-TET images are captured using the long TET, and wherein constructing the output image comprises:

forming a combined short-TET image, wherein the combined short-TET image includes the primary short-TET image and at least part of the one or more secondary short-TET images;

forming a combined long-TET image, wherein the combined long-TET image includes the primary long-TET image and at least part of the one or more secondary long-TET images; and

forming the output image, wherein the output image includes at least part of the combined short-TET image and at least part of the combined long-TET image.

4. The method of claim 1 , wherein the image sequence includes a long-TET image captured using the long TET, and wherein constructing the output image comprises:

determining that the long-TET image is a sharpest image of all images in the image sequence captured using the long TET;

based on the long-TET image being the sharpest image of all images in the image sequence captured using the long TET, selecting the long-TET image as a primary long-TET image; and

selecting, as a primary short-TET image, an image captured using the short TET that is adjacent to the primary long-TET image in the image sequence.

5. The method of claim 4 , wherein constructing the output image further comprises:

forming a combined short-TET image, wherein the combined short-TET image includes the primary short-TET image and at least part of one or more images from the image sequence that were captured using the short TET;

forming a combined long-TET image, wherein the combined long-TET image includes the primary long-TET image and at least part of one or more images from the image sequence that were captured using the long TET; and

forming the output image, wherein the output image includes at least part of the combined short-TET image and at least part of the combined long-TET image.

6. The method of claim 4 , wherein the primary short-TET image immediately precedes the primary long-TET image in the image sequence.

7. The method of claim 6 , wherein the short-TET image that immediately precedes the primary long-TET image in the image sequence is selected as the primary short-TET image based on image capture ending at the edge of respective image capture epochs.

8. The method of claim 4 , wherein the primary short-TET image immediately follows the primary long-TET image in the image sequence.

9. The method of claim 8 , wherein the short-TET image that immediately follows the primary long-TET image in the image sequence is selected as the primary short-TET image based on image capture beginning at the edge of respective image capture epochs.

10. An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by a computing device, cause the computing device to perform operations comprising:

capturing, by an image sensor, a first plurality of images of a scene, wherein each image of the first plurality of images is captured using a different total exposure time (TET);

based at least on the first plurality of images, determining a long TET, a short TET, and a TET sequence that includes the long TET and the short TET;

capturing, by the image sensor, a second plurality of images of the scene, wherein the images in the second plurality of images are captured sequentially in an image sequence using a sequence of TETs corresponding to the TET sequence,

wherein the image sequence includes a three-image sub-sequence of a first long-TET image captured using the long TET, followed by a short-TET image captured using the short TET, followed by a second long-TET image captured using the long TET; and

based on one or more images in the image sequence, constructing an output image.

11. The article of manufacture of claim 10 , wherein the image sequence also includes a second three-image sub-sequence of a first third long-TET image captured using the long TET, followed by a second fourth long-TET image captured using the long TET, followed by a second short-TET image captured using the short TET.

12. The article of manufacture of claim 10 , wherein the image sequence includes a two-image sub-sequence of a primary short-TET image followed by a primary long-TET image, wherein the image sequence also includes one or more secondary short-TET images and one or more secondary long-TET images, wherein the primary short-TET image and the secondary short-TET images are captured using the short TET, wherein the primary long-TET image and the secondary long-TET images are captured using the long TET, and wherein constructing the output image comprises:

forming a combined short-TET image, wherein the combined short-TET image includes the primary short-TET image and at least part of the one or more secondary short-TET images;

forming a combined long-TET image, wherein the combined long-TET image includes the primary long-TET image and at least part of the one or more secondary long-TET images; and

forming the output image, wherein the output image includes at least part of the combined short-TET image and at least part of the combined long-TET image.

13. The article of manufacture of claim 10 , wherein the image sequence includes a long-TET image captured using the long TET, and wherein constructing the output image comprises:

determining that the long-TET image is a sharpest image of all images in the image sequence captured using the long TET;

based on the long-TET image being the sharpest image of all images in the image sequence captured using the long TET, selecting the long-TET image as a primary long-TET image; and

selecting, as a primary short-TET image, an image captured using the short TET that is adjacent to the primary long-TET image in the image sequence.

14. The article of manufacture of claim 13 , wherein constructing the output image further comprises:

forming a combined short-TET image, wherein the combined short-TET image includes the primary short-TET image and at least part of one or more images from the image sequence that were captured using the short TET;

forming a combined long-TET image, wherein the combined long-TET image includes the primary long-TET image and at least part of one or more images from the image sequence that were captured using the long TET; and

forming the output image, wherein the output image includes at least part of the combined short-TET image and at least part of the combined long-TET image.

15. The article of manufacture of claim 13 , wherein the primary short-TET image immediately precedes the primary long-TET image in the image sequence.

16. A computing device comprising:

at least one processor;

an image sensor;

data storage; and

program instructions, stored in the data storage, that upon execution by the at least one processor cause the computing device to perform operations including:

capturing, by the image sensor, a first plurality of images of a scene, wherein each image of the first plurality of images is captured using a different total exposure time (TET);

based at least on the first plurality of images, determining a long TET, a short TET, and a TET sequence that includes the long TET and the short TET;

capturing, by the image sensor, a second plurality of images of the scene, wherein the images in the second plurality of images are captured sequentially in an image sequence using a sequence of TETs corresponding to the TET sequence, wherein the image sequence includes a three-image sub-sequence of a first long-TET image captured using the long TET, followed by a short-TET image captured using the short TET, followed by a second long-TET image captured using the long TET; and

based on one or more images in the image sequence, constructing an output image.

17. The computing device of claim 16 , wherein the image sequence also includes a second three-image sub-sequence of a first third long-TET image captured using the long TET, followed by a second fourth long-TET image captured using the long TET, followed by a second short-TET image captured using the short TET.

18. The computing device of claim 16 , wherein the image sequence includes a two-image sub-sequence of a primary short-TET image followed by a primary long-TET image, wherein the image sequence also includes one or more secondary short-TET images and one or more secondary long-TET images, wherein the primary short-TET image and the secondary short-TET images are captured using the short TET, wherein the primary long-TET image and the secondary long-TET images are captured using the long TET, and wherein constructing the output image comprises:

forming a combined short-TET image, wherein the combined short-TET image includes the primary short-TET image and at least part of the one or more secondary short-TET images;

forming a combined long-TET image, wherein the combined long-TET image includes the primary long-TET image and at least part of the one or more secondary long-TET images; and

forming the output image, wherein the output image includes at least part of the combined short-TET image and at least part of the combined long-TET image.

19. The computing device of claim 16 , wherein the image sequence includes a long-TET image captured using the long TET, and wherein constructing the output image comprises:

determining that the long-TET image is a sharpest image of all images in the image sequence captured using the long TET;

based on the long-TET image being the sharpest image of all images in the image sequence captured using the long TET, selecting the long-TET image as a primary long-TET image; and

selecting, as a primary short-TET image, an image captured using the short TET that is adjacent to the primary long-TET image in the image sequence.

20. The computing device of claim 19 , wherein constructing the output image further comprises:

forming a combined short-TET image, wherein the combined short-TET image includes the primary short-TET image and at least part of one or more images from the image sequence that were captured using the short TET;

forming a combined long-TET image, wherein the combined long-TET image includes the primary long-TET image and at least part of one or more images from the image sequence that were captured using the long TET; and

forming the output image, wherein the output image includes at least part of the combined short-TET image and at least part of the combined long-TET image.

21. A method comprising:

capturing, by an image sensor, a first plurality of images of a scene, wherein each image of the first plurality of images is captured using a different total exposure time (TET);

based at least on the first plurality of images, determining a long TET, a short TET, and a TET sequence that includes the long TET and the short TET;

capturing, by the image sensor, a second plurality of images of the scene, wherein the images in the second plurality of images are captured sequentially in an image sequence using a sequence of TETs corresponding to the TET sequence, wherein the image sequence includes a three-image sub-sequence of a first long-TET image captured using the long TET, followed by a second long-TET image captured using the long TET, followed by a short-TET image captured using the short TET; and

based on one or more images in the image sequence, constructing an output image.

22. A method comprising:

capturing, by an image sensor, a first plurality of images of a scene, wherein each image of the first plurality of images is captured using a different total exposure time (TET);

based at least on the first plurality of images, determining a long TET, a short TET, and a TET sequence that includes the long TET and the short TET;

capturing, by the image sensor, a second plurality of images of the scene, wherein the images in the second plurality of images are captured sequentially in an image sequence using a sequence of TETs corresponding to the TET sequence, wherein the image sequence includes a long-TET image captured using the long TET; and

based on one or more images in the image sequence, constructing an output image, wherein constructing the output image comprises (i) determining that the long-TET image is a sharpest image of all images in the image sequence captured using the long TET, (ii) based on the long-TET image being the sharpest image of all images in the image sequence captured using the long TET, selecting the long-TET image as a primary long-TET image, and (iii) selecting, as a primary short-TET image, an image captured using the short TET that is adjacent to the primary long-TET image in the image sequence.

Assignments (2)
CHANGE OF NAME Recorded Oct 2, 2017
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 044334/0466 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 13, 2012
From: GEISS, RYAN; HASINOFF, SAMUEL WILLIAM
To: GOOGLE INC.
Reel/Frame 029465/0131 →