IP Library Granted Patent US 8,559,705
Granted Patent B2
US 8,559,705 · App. 11/948,901 · Granted Oct 15, 2013

Interactive refocusing of electronic images

Inventor: Yi-Ren Ng (Mountain View, CA)
Assignee: Lytro, Inc.
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,559,705
App. No.
11/948,901
Granted
Oct 15, 2013
Kind
B2
Abstract

A method is performed to refocus a digital photographic image comprising a plurality of pixels. In the method, a set of images is computed corresponding to the digital photographic image and focused at different depths. Refocus depths for at least a subset of the pixels are identified and stored in a look-up table. At least a portion of the digital photographic image is refocused at a desired refocus depth determined from the look-up table.

Claims (57)

1. A method of refocusing a light field image in response to user input, the method comprising:

receiving a light field image comprising light field data;

digitally refocusing the light field image to generate a plurality of refocused images, each having a focus depth;

storing the generated plurality of refocused images in a storage device;

for each of a plurality of two-dimensional positions in the light field image:

determining which of the plurality of refocused images is in focus at the two-dimensional position; and

storing, in a look-up table, an index entry associating the two-dimensional position with the identified refocused image;

displaying one of the refocused images, the displayed image having a first focus depth;

receiving user input to select a two-dimensional position within the displayed image; and

in response to the received input:

retrieving, from the look-up table, an index entry identifying a stored refocused image corresponding to the selected two-dimensional position, the stored refocused image having a second focus depth different from the first focus depth;

retrieving the identified refocused image from the storage device; and

displaying the retrieved refocused image having the second focus depth.

2. The method of claim 1 , wherein receiving user input to select a two-dimensional position within the displayed image comprises:

displaying an icon to select a position within the displayed image;

receiving user input, via a pointing device, to move the displayed icon.

3. The method of claim 2 , wherein the icon comprises a pointer.

4. The method of claim 2 , wherein the icon comprises a brush tool to specify a region surrounding the selected position to refocus.

5. The method of claim 1 , wherein identifying a stored refocused image in the plurality having the second focus depth comprises:

identifying at least two of the refocused images having different focus depths; and

blending the at least two identified images.

6. The method of claim 1 , wherein receiving user input to select a two-dimensional position within the displayed image comprises:

detecting contact with a touch-sensitive screen, the contact specifying a position within the displayed image.

7. The method of claim 1 , wherein receiving user input to select a two-dimensional position within the displayed image comprises:

detecting a user interaction gesture specifying a position within the displayed image.

8. The method of claim 1 , further comprising:

displaying a smooth transition from the display of the image having the first focus depth to the display of at least one refocused image having an intermediate focus depth between the first focus depth and the second focus depth; and

displaying a smooth transition from the display of the at least one refocused image having an intermediate focus depth to the display of the refocused image having the second focus depth.

9. The method of claim 1 , wherein:

displaying the image having the first focus depth comprises:

retrieving a first stored image having the first focus depth; and

displaying the retrieved first image; and

displaying the refocused image having the second focus depth comprises:

retrieving a second stored image having the second focus depth; and

displaying the retrieved second image.

10. The method of claim 1 , wherein:

displaying one of the refocused images comprises displaying a web page comprising the refocused image;

receiving user input to select a two-dimensional position within the displayed image comprises receiving a signal indicating a user interaction with the web page; and

displaying the refocused image having the second focus depth comprises displaying a web page comprising the refocused image having the second focus.

11. The method of claim 1 , further comprising:

receiving an indication of at least one tag, each tag being associated with a position in the image, each tag being further associated with a description;

wherein:

receiving user input to select a two-dimensional position within the displayed image comprises receiving user input to select a description; and

displaying the refocused image having the second focus depth comprises refocusing the image to a focus depth corresponding to a position in the image associated with a tag associated with the selected description.

12. The method of claim 8 , wherein displaying a smooth transition from the display of the image having the first focus depth to the display of at least one refocused image having an intermediate focus depth comprises:

generating at least one intermediate image having a focus depth between the first focus depth and the second focus depth by determining an interpolated focus depth between the first focus depth and the second focus depth; and

displaying the generated an image at the interpolated focus depth.

13. The method of claim 1 , wherein digitally refocusing light field data to generate a plurality of refocused images comprises generating a plurality of projections of the light field image at different focus depths.

14. The method of claim 1 , wherein determining which of the plurality of refocused images is in focus at the two-dimensional position comprises detecting edges at the two-dimensional position for each of the refocused images.

15. The method of claim 1 , wherein determining which of the plurality of refocused images is in focus at the two-dimensional position comprises:

for each of the refocused images, computing a gradient magnitude at the two-dimensional position, each refocused image having a focus depth; and

comparing the computed gradient magnitudes at the two-dimensional position.

16. The method of claim 1 , wherein:

storing an index entry associating the two-dimensional position with identified refocused image comprises storing, in a look-up table, an index entry associating the two-dimensional position with the focus depth of the identified refocused image; and

identifying a stored refocused image in the plurality associated with the selected two-dimensional position comprises:

retrieving, from the look-up table, an index entry identifying a second focus depth corresponding to the selected two-dimensional position; and

identifying a stored refocused image in the plurality having the second focus depth.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 8, 2019
From: LYTRO, INC.
To: GOOGLE LLC
Reel/Frame 050009/0829 →
SECURITY INTEREST Recorded Mar 14, 2014
From: LYTRO, INC
To: TRIPLEPOINT CAPITAL LLC
Reel/Frame 032445/0362 →
CHANGE OF NAME Recorded Sep 27, 2013
From: REFOCUS IMAGING, INC.
To: LYTRO, INC.
Reel/Frame 031329/0348 →
SECURITY AGREEMENT Recorded Jan 31, 2013
From: LYTRO, INC.
To: TRIPLEPOINT CAPITAL LLC
Reel/Frame 029732/0787 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2008
From: NG, YI-REN
To: REFOCUS IMAGING, INC.
Reel/Frame 020604/0281 →
Continuity (2)
Provisional Application 60872089 · Dec 1, 2006
Related Publication 20080131019A1 · Jun 5, 2008