Method and an electronic device for calibrating a plenoptic camera
View Patent ↗The present disclosure concerns a method for calibrating a plenoptic camera. The plenoptic camera includes a lenslet array placed between a main lens and a photosensor, the lenslet array having a plurality of lenslets. The method is remarkable in that it includes obtaining a white image W and a model that describes light scene occlusion induced by the main lens and each lenslet. For a given pixel of the photosensor, where the model is represented by a set of parameters, obtaining a first subset of parameters values from the set of parameters in the model and obtaining a second subset of parameters value from the set of parameters in the model by performing a maximum likelihood estimation method based on the model, the white image and the first subset of parameters values, and where a union of the first and the second subsets of parameters corresponds to the set of parameters.
1. A method for calibrating a plenoptic camera, said plenoptic camera comprising a lenslet array placed between a main lens and a photosensor, said lenslet array comprising a plurality of lenslets, the method being characterized in that it comprises:
obtaining a white image W and a model that describes light scene occlusion induced by the main lens and each lenslet for a given pixel of said photosensor, wherein said model is represented by a set of parameters;
obtaining values of a first subset of parameters from said set of parameters in said model;
obtaining values of a second subset of parameters from said set of parameters in said model by performing a maximum likelihood estimation method based on said model, said white image and said values of a first subset of parameters, wherein a union of said first subset of parameters and said second subset of parameters corresponds to said set of parameters; and
applying the set of parameters to the plenoptic camera.
2. The method according to claim 1 , wherein, for said given pixel, said model specifies that it records a light intensity I(x p , y p ) according the following equation:
I
(
x
p
,
y
p
)
=
∫
x
=
-
∞
∞
∫
y
=
-
∞
∞
∫
μ
=
-
∞
∞
∫
v
=
-
∞
∞
L
(
x
+
λμ
,
y
+
λ
v
)
×
B
m
(
x
,
y
,
μ
,
v
)
B
l
(
x
-
c
x
l
d
,
y
-
c
y
l
d
)
rect
(
-
f
l
μ
+
x
p
p
,
-
f
l
v
+
y
p
p
)
ρ
(
f
l
-
1
x
+
μ
,
f
l
-
1
y
+
v
)
dv
d
μ
dy
dx
,
where function L(⋅, ⋅) is a source light field, and functions B m (⋅, ⋅, ⋅,⋅) and B l (⋅,⋅) are circular functions for modeling respectively occlusion of the main lens and an l-th lenslet, and function rect(⋅, ⋅) is a 2D rectangular function, and function ρ(⋅, ⋅) is an 2D function that describes an angular profile of photosensor, and where coordinates (x, y, μ, v) are coordinates induced by a two plane parametrization, a first plane being positioned at a photosensor level, and a second plane being positioned at a lenslet array level, and where parameter (c x l , c y l ) corresponds to the coordinates of the l-th lenslet'optical center, parameter f l corresponds to lenslet focal length, and where p corresponds to the photosensor pitch size, and where parameter λ is a depth parameter, and parameter d corresponds to a lenslet aperture size.
3. The method according to claim 2 , wherein said first subset of parameters comprises {F, d, p} and {c x m , c y m , {c x l , c y l } l=1 k }, where parameter F corresponds to the main lens F-number, parameter (c x m , c y m ) corresponds to the coordinates of the main lens optical center, and said second subset of parameters comprises {f, f l , σ}, where parameter f corresponds to a main lens focal length, and parameter σ corresponds to an angular exponent that is a parameter defining the function ρ.
4. The method according to claim 3 , wherein values of parameters comprised in {F, d, p} are known camera metadata.
5. The method according to claim 3 , wherein values of parameters comprised in {c x m , c y m , {c x l , c y l } l=1 k } are obtained from calibration techniques.
6. The method according to claim 2 , wherein functions B m (⋅, ⋅, ⋅, ⋅) and B l (⋅, ⋅) are defined according to the following equations:
B
m
(
x
,
y
,
μ
,
v
)
=
{
s
1
(
x
,
y
)
,
[
x
-
f
f
l
(
x
p
-
c
x
l
)
-
c
x
m
]
2
+
[
y
-
f
f
l
(
y
p
-
c
y
l
)
-
c
y
m
]
2
≤
(
D
2
)
2
0
,
otherwise
and
B
l
(
x
-
c
x
l
d
,
y
-
c
y
l
d
)
{
s
2
(
x
,
y
)
,
(
x
-
c
x
l
)
2
+
(
y
-
c
y
l
)
2
≤
(
d
2
)
2
0
,
otherwise
where s 1 (x, y) and s 2 (x, y) are functions that describe an effect on the luminance of incoming light rays due to the main lens and the lenslet, respectively, and where parameter D corresponds to a main lens aperture size, and where parameter (c x m , c y m ) corresponds to the coordinates of the main lens optical center, where parameter f corresponds to a main lens focal length, and where μ and v have the forms of
μ
=
x
p
-
c
x
l
f
l
and
v
=
y
p
-
c
y
l
f
l
.
7. The method according to claim 1 , wherein the performing of the maximum likelihood estimation method based on said model is done according to some constraints on said second subset of parameters.
8. An electronic device for calibrating a plenoptic camera, said plenoptic camera comprising a lenslet array placed between a main lens and a photosensor, said lenslet array comprising a plurality of lenslets, the electronic device comprising a memory, and at least one processor coupled to the memory, the at least one processor being characterized in that it is configured to:
obtain a white image W and a model that describes light scene occlusion induced by the main lens and each lenslet for a given pixel of said photosensor, wherein said model is represented by a set of parameters;
obtain values of a first subset of parameters from said set of parameters in said model;
obtain values of a second subset of parameters from said set of parameters in said model by performing a maximum likelihood estimation based on said model, said white image and values of said first subset of parameters, wherein a union of said first subset of parameters and said second subset of parameters corresponds to said set of parameters; and
apply the set of parameters to the plenoptic camera.