IP Library Granted Patent US 8,718,357
Granted Patent B2
US 8,718,357 · App. 13/717,061 · Granted May 6, 2014

Recovering 3D structure using blur and parallax

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,718,357
App. No.
13/717,061
Granted
May 6, 2014
Kind
B2
Abstract

A system and method for generating a focused image of an object is provided. The method comprises obtaining a plurality of images of an object, estimating an initial depth profile of the object, estimating a parallax parameter and a blur parameter for each pixel in the plurality of images and generating a focused image and a corrected depth profile of the object using a posterior energy function. The posterior energy function is based on the estimated parallax parameter and the blur parameter of each pixel in the plurality of images.

Claims (46)

1. A method for generating a focused image of an object from a first image of the object and a second image of the object, the method comprising:

estimating a parallax parameter based on motion of at least one pixel in the first image relative to the second image;

estimating a blur parameter based on a change in defocus of the at least one pixel from the first image to the second image;

determining a posterior energy function based on the parallax parameter and the blur parameter; and

generating at least one of a focused image of the object and a depth profile of the object using the posterior energy function.

2. The method of claim 1 , wherein a first portion of the object appears in focus in the first image and a second portion of the object appears in focus in the second image.

3. The method of claim 1 , wherein determining the posterior energy function comprises:

generating a blur map based on the blur parameter.

4. The method of claim 1 , further comprising:

obtaining at least one of the first image and the second image from a memory.

5. The method of claim 1 , further comprising:

obtaining the first image with an imaging system;

causing a change in distance between the object and a focal plane of the imaging system; and

obtaining the second image with the imaging system.

6. The method of claim 5 , wherein causing the change in distance between the object and the imaging system comprises:

moving at least one of the object and the imaging system along an optical axis of the imaging system.

7. The method of claim 5 , wherein the imaging system is member of a group consisting of: an endoscopy system, a laparoscopy system, and a digital photography system.

8. A system for obtaining a focused image of an object from a first image of the object and second image of the object, the system comprising:

a memory to store the first image and the second image; and

processing circuitry, operably coupled to the memory, to:

estimate a parallax parameter based on motion of at least one pixel in the first image relative to the second image;

estimate a blur parameter based on a change in defocus of the at least one pixel from the first image to the second image;

determine a posterior energy function based on the first and second parallax parameters and the first and second blur parameters; and

generate at least one of a focused image of the object and a corrected depth profile of the object using the posterior energy function.

9. The system of claim 8 , wherein a first portion of the object appears in focus in the first image and a second portion of the object appears in focus in the second image.

10. The system of claim 8 , wherein the processing circuitry is configured to determine the posterior energy function by generating a blur map based on the blur parameter.

11. The system of claim 8 , further comprising:

an imaging sensor, operably coupled to the memory, to acquire the first image and the second image.

12. The system of claim 11 , further comprising:

a lens to focus a first portion of the object on the imaging sensor during acquisition of the first image and to focus a second portion of the object on the imaging sensor during acquisition of the second image.

13. The system of claim 8 , wherein the system is member of a group consisting of: an endoscopy system, a laparoscopy system, and a digital photography system.

14. A system for generating a focused image of an object, comprising:

a memory to store a plurality of images of the object; and

processing circuitry, operably coupled to the memory, to:

select a subset of images from the plurality of images;

estimate a parallax parameter and a blur parameter for each pixel in the subset of images;

determine a posterior energy function based on the estimated parallax parameter and the blur parameter of each pixel in the plurality of images; and

generate at least one of a focused image and a depth profile of the object by applying the posterior energy function to the subset of images.

15. The system of claim 14 , wherein the subset of images includes a first image in which a first portion of the object appears in focus and a second image in which a second portion of the object appears in focus.

16. The system of claim 14 , wherein the blur parameter and the parallax parameter are a function of a structure of the object.

17. The system of claim 14 , wherein the processing circuitry is configured to apply a smoothness constraint to the subset of images.

18. The system of claim 17 , wherein the processing circuitry is further configured to obtain the smoothness constraint by using Gauss-Markov random fields.

19. The system of claim 14 , further comprising:

an imaging sensor to acquire the plurality of images; and

image acquisition circuitry, operably coupled to the imaging sensor and to the memory, to filter the plurality of images.

20. The system of claim 19 , wherein the plurality of images is obtained by moving the object relative to the imaging sensor over a plurality of incremental distances along an optical axis of the imaging sensor.

Assignments (4)
RELEASE OF SECURITY INTEREST IN PATENTS, RECORDED ON JANUARY 29, 2019 AT REEL 048373 FRAME 0217 Recorded Sep 22, 2025
From: CRESTLINE DIRECT FINANCE, L.P., AS COLLATERAL AGENT
To: EMPIRE TECHNOLOGY DEVELOPMENT LLC
Reel/Frame 072936/0464 →
RELEASE OF SECURITY INTEREST Recorded Jul 31, 2019
From: CRESTLINE DIRECT FINANCE, L.P.
To: EMPIRE TECHNOLOGY DEVELOPMENT LLC
Reel/Frame 049924/0794 →
SECURITY INTEREST Recorded Jan 29, 2019
From: EMPIRE TECHNOLOGY DEVELOPMENT LLC
To: CRESTLINE DIRECT FINANCE, L.P.
Reel/Frame 048373/0217 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 4, 2013
From: SAHAY, RAJIV RANJAN; NARAYANAN, RAJAGOPALAN AMBASAMUDRAM
To: INDIAN INSTITUTE OF TECHNOLOGY MADRAS
Reel/Frame 031535/0721 →