Method for obtaining a refocused image from 4D raw light field data
A method for obtaining a refocused image from 4D raw light field data for a given focus plane value g is described. The method is executed by an electronic device and is remarkable in that it comprises determining at least one pixel value, for at least one color component, of the refocused image at coordinates (k, l)∈N 2 . The at least one projected pixel value with coordinates comprised in a neighborhood of said coordinates (k, l), for the at least one color component is determined, weighted and updated.
1. A method for obtaining a refocused image from 4D raw light field data for a given focus plane value g, the method being executed by an electronic device, and wherein the method comprises determining at least one pixel value, for at least one color component, of said refocused image at coordinates (k, l)∈N 2 without using a demosaicing process on said 4D raw light field data, said determining comprising:
obtaining at least one projected pixel value with coordinates comprised in a neighborhood of said coordinates (k, l), for said at least one color component;
weighting said at least one projected pixel value based on a function, said weighting outputting a weighted value;
updating said at least one pixel value, for said at least one color component, of said refocused image at coordinates (k, l) with said weighted value.
2. The method for obtaining according to claim 1 , wherein said at least one projected pixel value has coordinates (X, Y), and said function is defined as being
W
1
(
k
,
l
,
X
,
Y
)
=
exp
(
-
(
k
,
l
)
-
(
X
,
Y
)
2
h
1
2
)
,
where h 1 corresponds to a value that belongs to R, and function∥.∥ is a norm function.
3. The method for obtaining according to claim 1 , wherein said at least one projected pixel value has coordinates (X, Y), and said function is defined as being
W
2
(
k
,
l
,
X
,
Y
)
=
exp
(
-
(
g
-
g
real
(
X
,
Y
)
)
2
h
2
2
)
,
where h 2 corresponds to a value that belongs to R, R, and value of g real (X, Y) corresponds to the real distance in a scene associated with said 4D raw light field data from a light field acquisition device that acquired said 4D raw light field data.
4. The method for obtaining according to claim 1 , wherein said at least one projected pixel value has coordinates (X, Y), and said function is defined as being
W
3
(
k
,
l
,
X
,
Y
)
=
exp
(
-
(
a
(
k
-
X
)
+
b
(
l
-
Y
)
)
2
h
3
2
)
exp
(
-
(
c
(
k
-
X
)
+
d
(
l
-
Y
)
)
2
h
3
′2
)
,
where h 3 and h 3 ′ correspond to a values that belong to R, and values a, b, c and d correspond to smoothing values determined according to tensor structure of said refocused image.
5. The method for obtaining according to claim 1 , wherein said at least one projected pixel value has coordinates (X, Y), and said function is defined as being W 6 (k, l, X, Y)=χ((k, l) belongs to the K−Nearest Neighbors pixels of ([X], [Y])), where χ( ) is the characteristic function which is equal to one if the statement is true and equal to zero otherwise, [.] is the rounding function, and K is an integer greater or equal to 2.
6. The method for obtaining according to claim 1 , wherein said at least one projected pixel value has coordinates (X, Y)=(s[g(x−C x )+C x ], s[g(y−C y )+C y ]) where, s defines the image size of the 2D refocused image, coordinates (x, y) correspond to coordinates of a pixel in said 4D raw light field data, coordinates (C x , C y ) correspond to coordinates of a center of a micro-image to which pixel at coordinates (x, y) belongs to, and said function is defined as being W 7 (k, l, X, Y)=PSF(x, y), where function PSF is an estimation of the Point Spread Function of a plenoptic camera.
7. The method for obtaining according to claim 1 , wherein said at least one projected pixel value has coordinates (X, Y)=(s[g(x−C x )+C x ], s[g(y−C y )+C y ]) where, s defines the image size of the 2D refocused image, coordinates (x, y) correspond to coordinates of a pixel in said 4D raw light field data, coordinates (C x , C y ) correspond to coordinates of a center of a micro-image to which pixel at coordinates (x, y) belongs to, and said function is defined as being W 8 =χ(∥(x, y)−(C x , C y )∥ 2 <ρ), where χ( ) is the characteristic function which is equal to one if the statement is true and equal to zero otherwise, ρ is a threshold value, and function∥.∥ is a norm function.
8. The method for obtaining according to claim 1 , wherein said at least one projected pixel value has coordinates (X, Y)=(s[g(x−C x )+C x ], s[g(y−C y )+C y ]) where, s defines the image size of the 2D refocused image, coordinates (x, y) correspond to coordinates of a pixel in said 4D raw light field data, coordinates (C x , C y ) correspond to coordinates of a center of a micro-image to which pixel at coordinates (x, y) belongs to, and said function is defined as being
W
9
(
k
,
l
,
X
,
Y
)
=
exp
(
-
(
x
,
y
)
-
(
C
x
,
C
y
)
2
h
8
2
)
,
where h 8 corresponds to a value that belongs to R, and function∥.∥ is a norm function.
9. The method for obtaining according to claim 2 , wherein said at least one projected pixel value has coordinates (X, Y)=(s[g(x−C x )+C x ], s[g(y−C y )+C y ]), where, s defines the image size of the 2D refocused image, coordinates (x, y) correspond to coordinates of a pixel in said 4D raw light field data, coordinates (C x , C y ) corresponds to coordinates of a center of a micro-image to which pixel at coordinates (x, y) belongs to, and said function is defined as being equal to
W
4
(
k
,
l
,
X
,
Y
)
=
1
S
(
x
,
y
)
-
S
max
W
i
with i≠4, where S(x, y) is a measure of sharpness of the microlens to which (x, y) belong and S max corresponds to a maximum sharpness value over all microlenses of said 4D raw light field data.
10. The method for obtaining according to claim 2 , wherein said function is defined as being separable.
11. The method for obtaining according to claim 2 , wherein said function is defined as being a product of
W
1
(
k
,
l
,
X
,
Y
)
=
exp
(
-
(
k
,
l
)
-
(
X
,
Y
)
2
h
1
2
)
,
where h 1 corresponds to a value that belongs to R, function∥.∥ is a norm function, with another function.
12. The method for obtaining according to claim 1 , wherein said updating comprises adding said weighted value to said at least one pixel value.
13. A computer-readable and non-transient storage medium storing a computer program comprising a set of computer-executable instructions to implement a method for processing 4D raw light field data when the instructions are executed by a computer, wherein the instructions comprise instructions, which when executed, configure the computer to perform a method for obtaining a refocused image from 4D raw light field data for a given focus plane value g, the method being executed by an electronic device, and wherein the method comprises determining at least one pixel value, for at least one color component, of said refocused image at coordinates (k, l)∈N 2 without using a demosaicing process on said 4D raw light field data, said determining comprising:
obtaining at least one projected pixel value with coordinates comprised in a neighborhood of said coordinates (k, l), for said at least one color component;
weighting said at least one projected pixel value based on a function, said weighting outputting a weighted value;
updating said at least one pixel value, for said at least one color component, of said refocused image at coordinates (k, l) with said weighted value.
14. An electronic device for obtaining a refocused image from 4D raw light field data for a given focus plane value g, wherein the electronic device comprises a circuit configured to determine at least one pixel value, for at least one color component, of said refocused image at coordinates (k, l)∈N 2 without using a demosaicing process on said 4D raw light field data, said circuit comprising:
an obtaining circuit configured to obtain at least one projected pixel value with coordinates comprised in a neighborhood of said coordinates (k, l), for said at least one color component;
a weighting circuit configured to weight said at least one projected pixel value based on a function, said weighting circuit outputting a weighted value;
an updating circuit configured to update said at least one pixel value, for said at least one color component, of said refocused image at coordinates (k, l) with said weighted value.
15. The electronic device according to claim 14 , wherein said at least one projected pixel value has coordinates (X, Y), and said function is defined as being
W
1
(
k
,
l
,
X
,
Y
)
=
exp
(
-
(
k
,
l
)
-
(
X
,
Y
)
2
h
1
2
)
,
where h 1 corresponds to a value that belongs to R, and function∥.∥ is a norm function.