Method and system for real-time noise removal and image enhancement of high-dynamic range images
A method and system for real-time noise removal and image enhancement of high-dynamic range (HDR) images. The method includes receiving an HDR input image I and operating processing circuitry for (i) applying a first edge-preserving filter (e.g. guided filter) to the input image I, thereby generating a first image component B 1 and a first set of linear coefficients ā i,1 ; (ii) applying a second edge-preserving filter (e.g. guided filter) to the input image I, thereby generating a second image component B 2 and a second set of linear coefficients ā i,2 ; (iii) generating a plausibility mask P from a combination of the first set of linear coefficients ā i,1 and the second set of linear coefficients ā i,2 , the plausibility mask P indicating spatial detail within the input image I; and (iv) generating an output image O based on first image component B 1 , the second image component B 2 and the plausibility mask P.
1. A method for real-time noise removal and image enhancement of high-dynamic range HDR images, the method comprising:
receiving an input image I, the input image I comprising a HDR image;
operating processing circuitry for
applying a first edge-preserving filter to the input image I, thereby generating a first image component B 1 and a first set of linear coefficients ā i,1 ;
said method is characterized by:
applying a second edge-preserving filter to the input image I, thereby generating a second image component B 2 and a second set of linear coefficients ā i,2 , wherein the first edge-preserving filter and the second edge-preserving filter are distinct filters having different filtering characteristics such that the first image component B 1 is different than the second image component B 2 , and the first set of linear coefficients ā i,1 is different than the second set of linear coefficients ā i,2 ;
generating a plausibility mask P from a combination of the first set of linear coefficients ā i,1 and the second set of linear coefficients ā i,2 , wherein the plausibility mask is derived as:
P ( i,j )= P′ ( i,j )− P′ min
where P′ =ā i,1 · ā i,2 and P′ min and P′ max the minimum and maximum weight values of P′, respectively; and
generating an output image O based on first image component B 1 , the second image component B 2 and the plausibility mask P.
2. The method of claim 1 , wherein the first edge-preserving filter comprises a detail preserving filter and the first image component B 1 comprises the input image I in which image detail is preserved.
3. The method of claim 1 , wherein the second edge-preserving filter comprises a noise removal filter and the second image component B 2 comprises the input image I from which noise has been removed or reduced.
4. The method of claim 1 , wherein the first edge-preserving filter and/or the second edge-preserving filter comprises a guided filter.
5. The method of claim 1 , further comprising cropping P′ before stretching its values according to
P
(
i
,
j
)
=
P
′
(
i
,
j
)
-
P
min
′
P
max
′
-
P
min
′
.
6. The method of claim 1 , further comprising cropping P′ according to
P
′
(
i
,
j
)
=
{
a
_
i
,
j
(
i
,
j
)
·
a
_
i
,
2
(
i
,
j
)
if
a
_
i
,
1
(
i
,
j
)
·
a
_
i
,
2
(
i
,
j
)
<
δ
δ
otherwise
.
where δ=μ+2σ, μ the mean of ā i,2 and σ=√{square root over (μ)}.
7. The method of claim 1 , wherein generating an output image O comprises generating an output image O based on the summation of the second image component B 2 and a detail image component D′ derived from the first image component B 1 and the plausibility mask P.
8. The method of claim 7 , wherein the detail image component D′=λP·D, where D=B 1 −B 2 , and λ is a gain factor for adjusting the contrast of details in D.
9. The method of claim 7 , wherein generating an output image O further comprises applying an intermediate image O′derived by
O′=B 2 +D′.
10. The method of claim 1 , wherein generating an output image O further comprises applying a mapping of an intermediate image O′ to output image O while preserving and enhancing the contrast of the image details in a detail image component D′.
11. The method of claim 10 , wherein the mapping comprises
generating a histogram of the grayscale value of each O′ (i,j) within the intermediate image O′.
12. The method of claim 10 , wherein the mapping comprises
classifying the grayscale value of each O′(i, j) within the intermediate image O′ as left O′ L (i,j) or right O′ R (i, j) depending on its location within the histogram of O′, such that
O′ L ( i,j )= O′ (i,j)·αif O′ ( i,j )·α<λ med
O′ R ( i,j )= O′ (i,j)·αif O′ ( i,j )·α≥λ med
where λ med corresponds to the location in the cumulative histogram of O′·α at 50% and a is a contrast level.
13. The method of claim 10 , wherein the mapping comprises determining output image O from
O
(
i
,
j
)
=
{
0
if
(
O
L
′
(
i
,
j
)
≤
λ
min
)
and
(
λ
med
-
λ
min
>
β
)
(
O
L
′
(
i
,
j
)
-
λ
min
)
·
β
λ
med
-
λ
min
if
(
O
L
′
(
i
,
j
)
>
λ
min
)
and
(
λ
med
-
λ
min
>
β
)
max
(
0
,
O
L
′
-
(
λ
med
-
β
)
)
if
(
λ
med
-
λ
min
≤
β
)
(
2
N
-
1
)
if
(
O
R
′
(
i
,
j
)
>
λ
max
)
and
(
λ
max
-
λ
med
>
(
2
N
-
1
)
-
β
)
(
O
R
′
(
i
,
j
)
-
λ
med
)
·
(
(
2
N
-
1
)
-
β
)
λ
max
-
λ
med
+
β
if
(
O
R
′
(
i
,
j
)
≤
λ
max
)
and
(
λ
max
-
λ
med
>
(
2
N
-
1
)
-
β
)
min
(
(
2
N
-
1
)
,
O
R
′
-
(
λ
med
-
β
)
)
if
(
λ
max
-
λ
med
≤
(
2
N
-
1
)
-
β
)
where λ min and λ max correspond to the boundaries of the cumulative histogram of O′·α, and N is the number of bits defining the dynamic range of output image O.
14. The method of claim 13 , wherein N=8 for 8 bit data representation within the range [0, 255].
15. The method of claim 9 , wherein the intermediate image O′ is a 12-bit or 14-bit image.
16. The method of claim 1 , wherein for the first image component B 1 and the second image component B 2 , each is derived using a linear transform of I in a window w k centered at pixel k, such that
B i =ā l I i + b v ∀iϵw k ,
where
a
ι
_
=
1
w
∑
k
∈
w
i
a
k
,
with
a
k
=
σ
k
2
σ
k
2
+
ɛ
,
b
ι
_
=
1
w
∑
k
∈
w
i
b
k
,
with
b
k
=
(
1
-
a
k
)
μ
k
,
are linear coefficients assumed to be constant in w k ,
μ k and σk 2 are the mean and variance of I in w k ,
|w| is the number of pixels in w k , and
ε is a regularization parameter penalizing large α k .
17. The method of claim 16 , wherein the size of w k is 3×3, ε 1 =25 and/or ε 2 =2500.
18. A programmable image processing system when suitably programmed for carrying out the method of claim 1 , the system comprising circuitry for receiving or storing an input image I and processing circuitry for performing the method.
19. A system for real-time noise removal and image enhancement of high-dynamic range HDR images, the system comprising:
circuitry for receiving or storing an input image I, the input image I comprising a HDR image;
processing circuitry, coupled to the circuitry for receiving or storing an input image I, the processing circuitry being operable for
applying a first edge-preserving filter to the input image I, thereby generating a first image component B 1 and a first set of linear coefficients ā i,1 ;
applying a second edge-preserving filter to the input image I, thereby generating a second image component B 2 and a second set of linear coefficients ā i,2 , wherein the first edge-preserving filter and the second edge-preserving filter are distinct filters having different filtering characteristics such that the first image component B 1 is different than the second image component B 2 , and the first set of linear coefficients ā i,1 is different than the second set of linear coefficients ā i,2 ;
generating a plausibility mask P from a combination of the first set of linear coefficients ā i,1 and the second set of linear coefficients ā i,2 , wherein the plausibility mask is derived as:
P ( i,j )= P′ ( i,j )− P′ min
where P′ =ā i,1 · ā i,2 and P′ min and P′ max the minimum and maximum weight values of P′, respectively; and
generating an output image O based on first image component B 1 , the second image component B 2 and the plausibility mask P.
20. A non-transitory computer-readable medium having recorded or stored thereon data defining or transformable into instructions for execution by processing circuitry and corresponding to at least the steps of claim 1 .
21. A server computer incorporating a communications device and a memory device and being adapted for transmission on demand or otherwise of data defining or transformable into instructions for execution by processing circuitry and corresponding to at least the steps of claim 1 .