IP Library › Granted Patent US 9,973,677
Granted Patent B2
US 9,973,677 · App. 14/053,436 · Granted May 15, 2018

Refocusable images

Inventors: Subramaniam Venkatraman (Fremont, CA); Victor Kulik (San Jose, CA)
Assignee: QUALCOMM Incorporated
H04N5/23212G06T7/00H04N5/2354H04N5/2356H04N5/23251H04N5/23258H04N5/23267H04N5/23293
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,973,677
App. No.
14/053,436
Granted
May 15, 2018
Kind
B2
Abstract

Embodiments of the present invention are directed toward providing refocusable image. Refocusable images are images that, after being captured, can be refocused, or adjusted to have a different focal length. The embodiments described herein can utilize a mobile user device, such as a smartphone, without the need for specialized hardware. A camera of the mobile device can be configured to take a series of images in succession over a short period of time in which the focal length of the camera is varied so that each image has a different focal length. An image stack is created by aligning the images, and images are processed to determine which image has the highest contrast (i.e., is in focus) for each region of the image stack. Embodiments can further include displaying a first image of the series of images.

Claims (56)

1. A method of image capture and processing, the method comprising:

capturing a plurality of images with a camera, wherein the capturing is triggered by at least one user input, and wherein the plurality of images include a stack of images capturing a scene over a period of time;

varying a focal length of the camera during the period of time such that each image of the stack of images is captured using a unique focal length;

processing, with a processing unit, the plurality of images by:

dividing each image of the stack of images into a plurality of regions, each region of the stack of images having a corresponding region in each image of the stack of images;

receiving a selection, from a first image in the stack of images, of a region including an object;

calculating a level of contrast for each region of a set of corresponding regions of the stack of images, the set of corresponding regions including the selected region;

comparing levels of contrast for regions of the set of corresponding regions;

determining a second image from the stack of images having a highest-level of contrast in the set of corresponding regions of the stack of images;

determining whether the object within the selected region is present at a different location in the second image than a location of the object in the first image; and

causing a display to show the second image with a notification that the object is present at the different location in at least the second image.

2. The method of claim 1 , wherein the selected region is selected from an initial displayed image of the plurality of images.

3. The method of claim 1 , wherein processing the plurality of images is performed by a computing device other than a mobile device.

4. The method of claim 1 , wherein dividing each image of the stack of images into the plurality of regions is based on a number of features detected in the stack of images, and wherein dividing each image.

5. The method of claim 1 , wherein dividing each image of the stack of images into the plurality of regions comprises dividing each image into regions of various sizes or shapes based on a number of features detected in the stack of images.

6. The method of claim 1 , wherein dividing each image of the stack of images into the plurality of regions comprises dividing each image into a grid, wherein a number of rows and a number of columns in the grid are determined based on a number of features detected in the stack of images.

7. The method of claim 1 , wherein the notification includes a text notification.

8. The method of claim 1 , further comprising causing the display to show at least a third image.

9. The method of claim 8 , further comprising causing the display to show the first image, the second image, and at least the third image, wherein the first image, the second image, and at least the third image include the object.

10. The method of claim 1 , further comprising:

removing, from the stack of images, at least one image in which none of the plurality of regions of the at least one image is determined to be a highest-contrast region among the corresponding regions of the stack of images.

11. An apparatus comprising:

a memory;

a display; and

a processing unit communicatively coupled with the memory, wherein the processing unit is configured to perform functions including:

obtaining a plurality of images captured by a camera of a mobile device, wherein capturing of the plurality of images is triggered by at least one user input, wherein the plurality of images include a stack of images capturing a scene over a period of time, and wherein a focal length of the camera is varied during the period of time such that each image of the stack of images is captured using a unique focal length;

processing the plurality of images by:

dividing each image of the stack of images into a plurality of regions, each region of the stack of images having a corresponding region in each image of the stack of images;

receiving a selection, from a first image in the stack of images, of a region including an object;

calculating a level of contrast for each region of a set of corresponding regions of the stack of images, the set of corresponding regions including the selected region;

comparing levels of contrast for regions of the set of corresponding regions;

determining a second image from the stack of images having a highest-level of contrast in the set of corresponding regions of the stack of images; and

determining whether the object within the selected region is present at a different location in the second image than a location of the object in the first image; and

causing the display to show a notification that the object is present at the different location in at least the second image.

12. The apparatus of claim 11 , wherein the apparatus comprises the mobile device.

13. The apparatus of claim 11 , wherein the notification includes a text notification.

14. The apparatus of claim 11 , wherein the processing unit is further configured to cause the display to show at least a third image.

15. The apparatus of claim 14 , wherein the processing unit is further configured to cause the display to show the first image, the second image, and at least the third image, wherein the first image, the second image, and at least the third image include the object.

16. The apparatus of claim 11 , wherein the processing unit is further configured to:

remove, from the stack of images, at least one image in which none of the plurality of regions of the at least one image is determined to be a highest-contrast region among the corresponding regions of the stack of images.

17. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to:

obtain a plurality of images captured with a camera of a mobile device, wherein capturing of the plurality of images is triggered by at least one user input, and wherein the plurality of images include a stack of images capturing a scene over a period of time;

cause the camera to vary a focal length during the period of time such that each image of the stack of images is captured using a unique focal length;

process the plurality of images by:

dividing each image of the stack of images into a plurality of regions, each region of the stack of images having a corresponding region in each image of the stack of images;

receiving a selection, from a first image in the stack of images, of a region including an object;

calculating a level of contrast for each region of a set of corresponding regions of the stack of images, the set of corresponding regions including the selected region;

comparing levels of contrast for regions of the set of corresponding regions;

determining a second image from the stack of images having a highest-level of contrast of the set in corresponding regions of the stack of images; and

determining whether the object within the selected region is present at a different location in the second image than a location of the object in the first image; and

cause a display to show a notification that the object is present at the different location in at least the second image.

18. The non-transitory computer-readable storage medium of claim 17 , further comprising instructions that, when executed by one or more processors, cause the one or more processors to align images of the stack of images, wherein aligning the plurality of images comprises at least one of:

scaling one or more images of the plurality of images,

shifting one or more images of the plurality of images,

rotating one or more images of the plurality of images, or

adjusting a brightness of one or more images of the plurality of images.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 17, 2014
From: VENKATRAMAN, SUBRAMANIAM; KULIK, VICTOR
To: QUALCOMM INCORPORATED
Reel/Frame 031992/0564 →
Continuity (1)
Related Publication 20150103192A1 · Apr 16, 2015