IP Library Granted Patent US 10,523,875
Granted Patent B2
US 10,523,875 · App. 16/217,704 · Granted Dec 31, 2019

Live updates for synthetic long exposures

Inventors: Samuel William Hasinoff (Mountain View, CA); Jiawen Chen (Mountain View, CA)
Assignee: Google Inc.
H04N5/2353H04N1/2137H04N5/2355H04N5/23216H04N5/23293H04N5/35581
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,523,875
App. No.
16/217,704
Granted
Dec 31, 2019
Kind
B2
Abstract

An image sensor of an image capture device may capture an image. The captured image may be stored in a buffer of two or more previously-captured images. An oldest image of the two or more previously-captured images may be removed from the buffer. An aggregate image of the images in the buffer may be updated. This updating may involve subtracting a representation of the oldest image from the aggregate image, and adding a representation of the captured image to the aggregate image. A viewfinder of the image capture device may display a representation of the aggregate image.

Claims (54)

1. A method comprising:

storing, in memory of an image capture device, a buffer of images and an aggregate image, wherein the aggregate image represents a first summation of pixel values in images stored in the buffer;

displaying, on a viewfinder of the image capture device, a representation of the aggregate image;

capturing, by an image sensor of the image capture device, a new image;

in response to capturing the new image, the image capture device: (i) removing, from the buffer, an oldest image, (ii) storing, in the buffer, the new image, and (iii) updating the aggregate image to represent a second summation of pixel values in the images stored in the buffer; and

displaying, on the viewfinder, a representation of the aggregate image as updated.

2. The method of claim 1 , wherein updating the aggregate image comprises:

subtracting the oldest image from the aggregate image; and

adding the new image to the aggregate image.

3. The method of claim 1 , wherein capturing the new image occurs while the viewfinder displays the representation of the aggregate image.

4. The method of claim 1 , wherein the viewfinder has a refresh rate defined by a refresh time interval, wherein the representation of the aggregate image has a synthetic exposure length defined by a synthetic exposure time interval, wherein the synthetic exposure time interval is greater than the refresh time interval.

5. The method of claim 4 , wherein synthetic exposure time interval is at least 1000 milliseconds and the refresh time interval is less than 150 milliseconds.

6. The method of claim 4 , wherein the synthetic exposure time interval is an integer multiple of the refresh time interval.

7. The method of claim 6 , further comprising:

providing, by the image capture device, one or more user interface components configured to receive a selection of the synthetic exposure time interval from a plurality of possible synthetic exposure time intervals that are different integer multiples of the refresh time interval; and

setting the synthetic exposure time interval as selected.

8. The method of claim 4 , wherein the synthetic exposure time interval is a real-number multiple of the refresh time interval.

9. The method of claim 8 , further comprising:

providing, by the image capture device, one or more user interface components configured to receive a selection of the synthetic exposure time interval from a plurality of possible synthetic exposure time intervals that are different real-number multiples of the refresh time interval; and

setting the synthetic exposure time interval as selected.

10. The method of claim 8 , wherein n represents the synthetic exposure time interval, wherein f represents a total length of all whole refresh time intervals in the synthetic exposure time interval, and wherein capturing the new image comprises:

applying a gain of approximately 1+(n−f)/f to the new image.

11. The method of claim 1 , wherein the buffer stores at least 15 captured images in time order of capture.

12. The method claim 1 , wherein the representation of the aggregate image is a low-resolution image, and wherein previously-captured images stored in the buffer and the aggregate image are high-resolution images that have higher resolutions than the low-resolution image, the method further comprising:

receiving, by the image capture device, an indication that a shutter function has been triggered; and

storing, in the memory of the image capture device, the aggregate image, such that the aggregate image is available to one or more image manipulation applications.

13. The method of claim 1 , wherein the image capture device includes an image processing pipeline that applies tone mapping to captured images, and wherein the new image is stored in the buffer without image processing from the image processing pipeline.

14. The method of claim 1 , wherein the image capture device includes an image processing pipeline that applies tone mapping to captured images before storing the captured images in the buffer, the method further comprising:

after tone mapping is applied and before storing the new image in the buffer, applying reverse tone mapping to the new image, wherein the reverse tone mapping removes the tone mapping from the new image.

15. The method of claim 1 , wherein updating the aggregate image comprises applying image stabilization to the aggregate image.

16. An article of manufacture including a non-transitory computer-readable medium, having stored thereon program instructions that, upon execution by an image capture device, cause the image capture device to perform operations comprising:

storing, in memory of the image capture device, a buffer of images and an aggregate image, wherein the aggregate image represents a first summation of pixel values in images stored in the buffer;

displaying, on a viewfinder of the image capture device, a representation of the aggregate image;

capturing, by an image sensor of the image capture device, a new image;

in response to capturing the new image, (i) removing, from the buffer, an oldest image, (ii) storing, in the buffer, the new image, and (iii) updating the aggregate image to represent a second summation of pixel values in the images stored in the buffer; and

displaying, on the viewfinder, a representation of the aggregate image as updated.

17. The article of manufacture of claim 16 , wherein the viewfinder has a refresh rate defined by a refresh time interval, wherein the representation of the aggregate image has a synthetic exposure length defined by a synthetic exposure time interval, wherein the synthetic exposure time interval is greater than the refresh time interval.

18. The article of manufacture of claim 17 , wherein the synthetic exposure time interval is an integer multiple of the refresh time interval, the operations further comprising:

providing one or more user interface components configured to receive a selection of the synthetic exposure time interval from a plurality of possible synthetic exposure time intervals that are different integer multiples of the refresh time interval; and

setting the synthetic exposure time interval as selected.

19. The article of manufacture of claim 17 , wherein the synthetic exposure time interval is a real-number multiple of the refresh time interval, the operations further comprising:

providing one or more user interface components configured to receive a selection of the synthetic exposure time interval from a plurality of possible synthetic exposure time intervals that are different real-number multiples of the refresh time interval; and

setting the synthetic exposure time interval as selected.

20. An image capture device comprising:

an image sensor;

a viewfinder;

a processor;

a memory; and

program instructions, stored in the memory, that upon execution by the processor cause the image capture device to perform operations comprising:

storing, in the memory, a buffer of images and an aggregate image, wherein the aggregate image represents a first summation of pixel values in images stored in the buffer;

displaying, on the viewfinder, a representation of the aggregate image;

capturing, by the image sensor, a new image;

in response to capturing the new image, (i) removing, from the buffer, an oldest image, (ii) storing, in the buffer, the new image, and (iii) updating the aggregate image to represent a second summation of pixel values in the images stored in the buffer; and

displaying, on the viewfinder, a representation of the aggregate image as updated.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 13, 2018
From: HASINOFF, SAMUEL WILLIAM; CHEN, JIAWEN
To: GOOGLE INC.
Reel/Frame 047764/0912 →
CHANGE OF NAME Recorded Dec 13, 2018
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 049139/0397 →
Continuity (2)
Continuation 15097687 · Apr 13, 2016
Related Publication 20190116304A1 · Apr 18, 2019
Cited By (1)
US 12,439,163