Recovering 3D structure using blur and parallax
View Patent ↗A system and method for generating a focused image of an object is provided. The method comprises obtaining a plurality of images of an object, estimating an initial depth profile of the object, estimating a parallax parameter and a blur parameter for each pixel in of the plurality of images and generating a focused image and a corrected depth profile of the object using a posterior energy function. The posterior energy function is based on the estimated parallax parameter and the blur parameter of each pixel in the plurality of images.
1. A method for generating a focused image of an object, comprising:
obtaining a plurality of images of the object;
estimating an initial depth profile of the object;
estimating a parallax parameter and a blur parameter for each pixel in each of the plurality of images; and
generating a focused image and a corrected depth profile of the object using a posterior energy function, wherein the posterior energy function is based on the estimated parallax parameter and blur parameter for each pixel in the plurality of images.
2. The method of claim 1 , further comprising selecting a subset of images from the plurality of images.
3. The method of claim 2 , wherein the posterior energy function is applied to the subset of images.
4. The method of claim 1 , wherein the posterior energy function comprises applying a smoothness constraint.
5. The method of claim 4 , wherein the smoothness constraint is obtained by using Gauss-Markov random fields.
6. The method of claim 1 , wherein the blur parameter and the parallax parameter are a function of a structure of the object.
7. The method of claim 1 , wherein the plurality of images are obtained by moving the object over a plurality of incremental distances along one axis.
8. The method of claim 1 , wherein the posterior energy function is represented by the following relation:
U
p
(
d
_
,
x
)
=
∑
m
ɛ
O
y
m
-
H
m
(
d
_
)
W
m
(
d
_
)
x
2
2
σ
η
2
+
λ
d
_
∑
c
ɛ
C
d
_
V
c
d
_
(
d
_
)
+
λ
x
∑
c
ɛ
C
x
(
V
c
x
)
(
x
)
where ‘x’ is the focused image, d is the depth profile, ‘y m ’ is a lexicographically arranged vector of size M×M×1 derived from an m th defocused and scaled image, W m describes the motion of the pixels in the m th image, H m is a blur map for the m th image, and n m is the Gaussian noise vector in the m th image, c′ is a clique, ‘C’ is the set of all cliques, V c d ( d ) and V c x (x) are potentials associated with clique ‘c’, σ η 2 is the Gaussian noise, λ d and λ x are smoothness parameters.
9. The method of claim 1 , wherein the step of obtaining a plurality of images comprises obtaining at least two images.
10. A system for obtaining a focused image of an object, the system comprising:
an image sensor configured to obtain a plurality of images of the object, processing circuitry configured to:
estimate an initial depth profile of the object,
estimate a parallax parameter and a blur parameter for each pixel in of the plurality of images; and
generate a focused image and a corrected depth profile of the object using a posterior energy function, wherein the posterior energy function is based on the estimated parallax parameter and the blur parameter of each pixel in the plurality of images.
11. The system of claim 10 , further comprising a display unit configured to display the plurality of images to enable a user to select a subset of images from the plurality of images.
12. The system of claim 11 , wherein the posterior energy function is applied to the subset of images.
13. The system of claim 10 , wherein the posterior energy function is represented by the following relation:
U
p
(
d
_
,
x
)
=
∑
m
ɛ
O
y
m
-
H
m
(
d
_
)
W
m
(
d
_
)
x
2
2
σ
η
2
+
λ
d
_
∑
c
ɛ
C
d
_
V
c
d
_
(
d
_
)
+
λ
x
∑
c
ɛ
C
x
(
V
c
x
)
(
x
)
where ‘x’ is the focused image, d is the depth profile, ‘y m ’ is a lexicographically arranged vector of size M×M×1 derived from an m th defocused and scaled image, W m describes the motion of the pixels in the m th image, H m is a blur map for the m th image, and n m is the Gaussian noise vector in the m th image, c′ is a clique, ‘C’ is the set of all cliques, V c d ( d ) and V c x (x) are potentials associated with clique ‘c’, σ η 2 is the Gaussian noise, λ d and λ x are smoothness parameters.
14. The system of claim 10 , wherein the blur parameter and the parallax parameter are a function of a three dimensional structure of the object.
15. The system of claim 10 , wherein the plurality of images are obtained by moving the object over a plurality of incremental distances along one axis.
16. A method for generating a focused image of an object, comprising:
obtaining a plurality of images of an object,
estimating an initial depth profile of the object,
selecting a subset of images from the plurality of images;
estimating a parallax parameter and a blur parameter for each pixel in of the subset of images; and
generating a focused image and a corrected depth profile of the object by applying a posterior energy function on the subset of images, wherein the posterior energy function is based on the estimated parallax parameter and the blur parameter of each pixel in the plurality of images.
17. The method of claim 16 , wherein the posterior energy function comprises applying a smoothness constraint.
18. The method of claim 17 , wherein the smoothness constraint is obtained by using Gauss-Markov random fields.
19. The method of claim 16 , wherein the blur parameter and the parallax parameter are a function of a structure of the object.
20. The method of claim 16 , wherein the plurality of images are obtained by translating the object on one axis over a plurality of incremental distances.