IP Library Granted Patent US 10,110,867
Granted Patent B2
US 10,110,867 · App. 15/808,753 · Granted Oct 23, 2018

Systems and methods for digital photography

Inventors: William Guie Rivard (Menlo Park, CA); Adam Barry Feder (Mountain View, CA); Brian J. Kindle (Sunnyvale, CA)
Assignee: DUELIGHT LLC
H04N9/735H04N1/3871H04N5/2256H04N5/2353H04N5/2354H04N5/23232H04N9/045H04N9/76
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,110,867
App. No.
15/808,753
Granted
Oct 23, 2018
Kind
B2
Abstract

A system, method, and computer program product are provided for displaying a combined image based on a cost function. In use, two or more source images are loaded and a first blend weight is initiated associated with the two or more source images. Next, a first combined image from the two or more source images and a cost function for the first combined image is computed. It is determined whether the cost function is substantially minimized, wherein if the cost function is not substantially minimized, the first blend weight is updated to substantially minimize the cost function, the first combined image is updated based on the updated first blend weight, and the user interface element associated with the updated first combined image is updated. A display then displays the first combined image based on the first blend weight and a user interface element associated with the first combined image. Additional systems, methods, and computer program products are also presented.

Claims (65)

1. A device, comprising:

a non-transitory memory storing instructions; and

one or more processors in communication with the non-transitory memory, wherein the one or more processors execute the instructions to:

sample a first ambient image and a first strobe image;

generate an aligned ambient image based on the first ambient image;

generate an aligned strobe image based on the first strobe image;

blend the aligned ambient image and the aligned strobe image by:

initialing a first blend weight associated with the aligned ambient image and the aligned strobe image;

rendering a first combined image from the aligned ambient image and the aligned strobe image;

computing a cost function for the first combined image, wherein the cost function optimizes a pixel intensity associated with each of the aligned ambient image and the aligned strobe image;

determining whether the cost function is substantially minimized, wherein if the cost function is not substantially minimized:

the first blend weight is updated to substantially minimize the cost function,

the first combined image is updated based on the updated first blend weight, and

the user interface element associated with the updated first combined image is updated;

display, using a display:

the first combined image based on the first blend weight, and

a user interface element associated with the first combined image.

2. The device of claim 1 , wherein the device is configured such that the cost function is computed by interpolating one or more attributes for each of the aligned ambient image and the aligned strobe image.

3. The device of claim 2 , wherein the device is configured such that the one or more attributes includes at least one of a low intensity mark, a high intensity mark, or a half intensity value.

4. The device of claim 3 , wherein the device is configured such that the cost function is computed as a sum of a distance from a low intensity mark to a first half intensity value, and a distance from a high intensity mark and a second half intensity value.

5. The device of claim 1 , wherein the device is configured such that the cost function is computed from a plurality of median values from each of the aligned ambient image and the aligned strobe image.

6. The device of claim 1 , wherein the device is configured such that the user interface element includes a control element.

7. The device of claim 1 , wherein the device is configured such that a position of the control element includes an offset estimate based on a user preference or a history of control element overrides.

8. The device of claim 1 , wherein the device is configured such that the first blend weight is a function of the user interface element and modifying the user interface element overrides the updated first blend weight.

9. The device of claim 1 , wherein the device is configured such that the cost function is proportional to an area of the aligned ambient image and the aligned strobe image.

10. The device of claim 9 , wherein the device is configured such that the area includes a subset of the aligned ambient image and the aligned strobe image that is either overexposed or underexposed.

11. The device of claim 10 , wherein the device is configured such that the cost function is based on a sum where each pixel within the combined image adds a value to the cost function if a pixel intensity is below a low threshold or above a high threshold.

12. The device of claim 10 , wherein the device is configured such that the cost function is based on a sum where each pixel within the combined image adds an increasing value to the cost function in proportion to pixel intensity.

13. The device of claim 12 , wherein the device is configured such that as the pixel intensity increases above a high threshold or as the pixel intensity decreases below a low threshold, an increasing cost value is added to the cost function.

14. The device of claim 13 , wherein the device is configured such that the high threshold is ninety percent (90%) of maximum defined intensity for each pixel and the low threshold is ten percent (10%) of the maximum defined intensity for each pixel.

15. The device of claim 1 , wherein the device is configured such that the cost function includes a repulsion cost component, wherein the repulsion cost increases the cost function as the user interface element approaches a preconfigured anchor point of the user interface element.

16. The device of claim 1 , wherein the device is configured such that the cost function includes an attraction cost component, wherein the attraction cost decreases the cost function as the user interface element approaches a preconfigured anchor point of the user interface element.

17. The device of claim 1 , wherein the device is configured such that the first blend weight is based on exposure coordinates of the aligned ambient image and the aligned strobe image, the exposure coordinates being selected based on searching the aligned ambient image and the aligned strobe image for a coordinate that satisfies a corresponding exposure requirement.

18. The device of claim 1 , wherein the device is configured such that the cost function is substantially minimized by being minimized.

19. The device of claim 1 , wherein the device is configured such that the cost function is substantially minimized by being completely minimized.

20. A computer program product comprising computer executable instructions stored on a non-transitory computer readable medium that when executed by a processor instruct the processor to:

sample a first ambient image and a first strobe image;

generate an aligned ambient image based on the first ambient image;

generate an aligned strobe image based on the first strobe image;

blend the aligned ambient image and the aligned strobe image by:

initializing a first blend weight associated with the aligned ambient image and the aligned strobe image;

rendering a first combined image from the aligned ambient image and the aligned strobe image;

computing a cost function for the first combined image, wherein the cost function optimizes a pixel intensity associated with each of the aligned ambient image and the aligned strobe image;

determining whether the cost function is substantially minimized, wherein if the cost function is not substantially minimized:

the first blend weight is updated to substantially minimize the cost function,

the first combined image is updated based on the updated first blend weight, and

the user interface element associated with the updated first combined image is updated

display, using a display:

the first combined image based on the first blend weight, and

a user interface element associated with the first combined image.

21. A computer-implemented method, comprising:

sampling, using an image sensor, a first ambient image and a first strobe image;

generating, using a processor, an aligned ambient image based on the first ambient image;

generating, using the processor, an aligned strobe image based on the first strobe image;

blending the aligned ambient image and the aligned strobe image by:

initializing, using the processor, a first blend weight associated with the aligned ambient image and the aligned strobe image;

rendering, using the processor, a first combined image from the aligned ambient image and the aligned strobe image;

computing, using the processor, a cost function for the first combined image, wherein the cost function optimizes a pixel intensity associated with each of the aligned ambient image and the aligned strobe image;

determining, using the processor, whether if the cost function is substantially minimized, wherein if the cost function is not substantially minimized:

the first blend weight is updated to substantially minimize the cost function,

the first combined image is updated based on the updated first blend weight, and

the user interface element associated with the updated first combined image is updated

displaying, using a display:

the first combined image based on the first blend weight, and

a user interface element associated with the first combined image.

Assignments (2)
SECURITY INTEREST Recorded Jul 30, 2021
From: DUELIGHT, LLC
To: ZILKA KOTAB, P.C.
Reel/Frame 057043/0301 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 20, 2018
From: RIVARD, WILLIAM GUIE; FEDER, ADAM BARRY; KINDLE, BRIAN J.
To: DUELIGHT LLC
Reel/Frame 045603/0485 →
Continuity (4)
Continuation 14887211 · Oct 19, 2015
Continuation 13999343 · Feb 11, 2014
Provisional Application 61850246 · Feb 12, 2013
Related Publication 20180070069A1 · Mar 8, 2018
Cited By (5)
US 12,401,911 US 12,401,912 US 12,418,727 US 12,445,736 US 12,666,159