Image processing method, storage medium, and terminal
The present disclosure provides an image processing method, device, terminal, and storage medium. The method includes: obtaining an RGB-NIR image sensor replacing a G component in a Bayer RGGB mode with an NIR component; using the RGB-NIR image sensor to obtain an RGB-NIR RAW image; and obtaining an RGB image and a near-infrared (NIR) image by demosaicing the RGB-NIR RAW image based on a vector median method.
1. An image processing method, comprising:
providing an RGB-NIR image sensor with an RGB-NIR filter pattern obtained by replacing a G component in a Bayer RGGB mode with an NIR component;
using the RGB-NIR image sensor to obtain an RGB-NIR raw image; and
based on a vector median method, obtaining an RGB image and a near-infrared (NIR) image by demosaicing the RGB-NIR raw image, including:
using spectral color information for a first pixel set to form an N-dimensional vector of a first pixel, wherein the first pixel is each pixel in the RGB-NIR RAW image, the first pixel set contains pixels around the first pixel, and the N is a total number of spectral color components of the RGB-NIR RAW image;
according to an element of each dimension vector in the N-dimensional vector, determining a corresponding vector median;
forming an N-dimensional median vector x={x 1 , . . . x n } by N medians corresponding to the N-dimensional vector of the first pixel, wherein x i stands for a median of i-th vector in the N-dimensional vector, 1≤i≤N; and
according to
r
=
x
n
2
+
…
+
x
1
2
,
φ
1
=
arc
cot
x
1
x
2
2
+
…
+
x
n
2
,
…
φ
n
-
2
=
arc
cot
x
n
-
2
x
n
-
1
2
+
x
n
2
,
φ
n
-
1
=
2
arc
cot
x
n
-
1
2
+
x
n
2
+
x
n
-
1
x
n
,
translating x={x 1 , . . . x n } into y={r, ϕ 1 , . . . ϕ n-1 };
using 2-dimensional median filtering to process an (N−1)-dimensional vector {ϕ 1 , . . . ϕ n-1 };
according to
x
_
1
=
r
cos
(
ϕ
1
)
,
x
_
2
=
r
sin
(
ϕ
1
)
cos
(
ϕ
2
)
,
…
x
_
n
-
1
=
r
sin
(
ϕ
1
)
…
sin
(
ϕ
n
-
2
)
cos
(
ϕ
n
-
1
)
,
x n-1 =r sin(ϕ 1 ) . . . sin(ϕ n-2 )sin(ϕ n-1 ), calculating a filtered value in an N-dimensional Euclid space to obtain x ={ x 1 , . . . , x n } as a final pixel vector.
2. The method according to claim 1 , wherein determining the pixels around the first pixel further comprises:
based on a horizontal and diagonal searching format, or a vertical and horizontal searching format, determining the pixels around the first pixel.
3. The method according to claim 1 , wherein using the spectral color information for the first pixel set to form the N-dimensional vector of the first pixel further comprises:
based on a horizontal and diagonal searching format, or a vertical and horizontal searching format, determining the first pixel set;
obtaining the spectral color information for the first pixel set; and
using the spectral color information for the first pixel set to form a vector of each spectral color component of the first pixel.
4. The method according to claim 1 , wherein, according to an element of each dimension vector in the N-dimensional vector, determining the corresponding vector median further comprises:
for the element of each dimension vector x i , calculating a one-dimensional distance
S
i
1
=
∑
j
=
1
N
x
i
-
x
j
1
,
i
=
1
,
…
N
and a two-dimensional distance
S
i
2
=
∑
j
=
1
N
x
i
-
x
j
2
,
i
=
1
,
…
N
between the element and a neighbouring element, and counting a sum S i of the one-dimensional distance and the two-dimensional distance as S i =S 1 i +S 2 i ; and
determining the element x i corresponding to a minimum value in the S i as the corresponding vector median.
5. A non-transitory computer-readable storage medium, storing computer-executable instructions for execution by a processor configured to:
use an RGB-NIR image sensor to obtain an RGB-NIR raw image, wherein the RGB-NIR image sensor is provided with an RGB-NIR filter pattern obtained by replacing a G component in a Bayer RGGB mode with an NIR component; and
obtain an RGB image and a near-infrared (NIR) image by demosaicing the RGB-NIR raw image based on a vector median method, wherein the processor is further configured to:
use spectral color information for a first pixel set to form an N-dimensional vector of a first pixel, wherein the first pixel is each pixel in the RGB-NIR RAW image, the first pixel set contains pixels around the first pixel, and the N is a total number of spectral color components of the RGB-NIR RAW image;
according to an element of each dimension vector in the N-dimensional vector, determine a corresponding vector median;
form an N-dimensional median vector x={x 1 , . . . x n } by N medians corresponding to the N-dimensional vector of the first pixel, wherein x i stands for a median of i-th vector in the N-dimensional vector, 1≤i≤N;
according to
r
=
x
n
2
+
…
+
x
1
2
,
φ
1
=
arc
cot
x
1
x
2
2
+
…
+
x
n
2
,
…
φ
n
-
2
=
arc
cot
x
n
-
2
x
n
-
1
2
+
x
n
2
,
φ
n
-
1
=
2
arc
cot
x
n
-
1
2
+
x
n
2
+
x
n
-
1
x
n
,
translate x={x 1 , . . . x n } into y={r, ϕ 1 , . . . ϕ n-1 };
use 2-dimensional median filtering to process an (N−1)-dimensional vector {ϕ 1 , . . . ϕ n-1 }; and
according to
x
_
1
=
r
cos
(
ϕ
1
)
,
x
_
2
=
r
sin
(
ϕ
1
)
cos
(
ϕ
2
)
,
…
x
_
n
-
1
=
r
sin
(
ϕ
1
)
…
sin
(
ϕ
n
-
2
)
cos
(
ϕ
n
-
1
)
,
x
_
n
-
1
=
r
sin
(
ϕ
1
)
…
sin
(
ϕ
n
-
2
)
sin
(
ϕ
n
-
1
)
,
calculate a filtered value in an N-dimensional Euclid space to obtain x ={ x 1 , . . . x n } as a final pixel vector.
6. The storage medium according to claim 5 , wherein the processor is configured to determine the pixels around the first pixel by:
based on a horizontal and diagonal searching format, or a vertical and horizontal searching format, determining the pixels around the first pixel.
7. The storage medium according to claim 5 , wherein the processor is further configured to:
determine the first pixel set based on a horizontal and diagonal searching format or a vertical and horizontal searching format;
obtain the spectral color information for the first pixel set; and
use the spectral color information for the first pixel set to form a vector of each spectral color component of the first pixel.
8. The storage medium according to claim 5 , wherein the processor is further configured to:
calculate a one-dimensional distance
S
i
1
=
∑
j
=
1
N
x
i
-
x
j
1
,
i
=
1
,
…
N
and a two-dimensional distance
S
i
2
=
∑
j
=
1
N
x
i
-
x
j
2
,
i
=
1
,
…
N
between the element of each dimension vector x i and a neighbouring element, and count a sum S i of the one-dimensional distance and the two-dimensional distance as S i =S 1 i +S 2 i ; and
determine the element x i corresponding to a minimum value in the S i as the corresponding vector median.
9. A terminal, comprising:
an RGB-NIR image sensor, a memory and a processor, wherein:
the RGB-NIR image sensor is provided with an RGB-NIR filter pattern obtained by replacing a G component in a Bayer RGGB mode with an NIR component and is configured to obtain an RGB-NIR raw image;
the memory is configured to store the RGB-NIR raw image; and
the processor is configured to obtain an RGB image and a near-infrared (NIR) image by demosaicing the RGB-NIR raw image based on a vector median method, wherein the processor is further configured to:
use spectral color information for a first pixel set to form an N-dimensional vector of a first pixel, wherein the first pixel is each pixel in the RGB-NIR RAW image, the first pixel set contains pixels around the first pixel, and the N is a total number of spectral color components of the RGB-NIR RAW image;
according to an element of each dimension vector in the N-dimensional vector, determine a corresponding vector median;
form an N-dimensional median vector x={x 1 , . . . x n } by N medians corresponding to the N-dimensional vector of the first pixel, wherein x i stands for a median of i-th vector in the N-dimensional vector, 1≤i≤N;
according to
r
=
x
n
2
+
…
+
x
1
2
,
φ
1
=
arc
cot
x
1
x
2
2
+
…
+
x
n
2
,
…
φ
n
-
2
=
arc
cot
x
n
-
2
x
n
-
1
2
+
x
n
2
,
φ
n
-
1
=
2
arc
cot
x
n
-
1
2
+
x
n
2
+
x
n
-
1
x
n
,
translate x={x 1 , . . . x n } into y={r, ϕ 1 , . . . ϕ n-1 };
use 2-dimensional median filtering to process an (N−1)-dimensional vector {ϕ 1 , . . . ϕ n-1 };
according to
x
_
1
=
r
cos
(
ϕ
1
)
,
x
_
2
=
r
sin
(
ϕ
1
)
cos
(
ϕ
2
)
,
…
x
_
n
-
1
=
r
sin
(
ϕ
1
)
…
sin
(
ϕ
n
-
2
)
cos
(
ϕ
n
-
1
)
,
x n-1 =r sin(ϕ 1 ) . . . sin(ϕ n-2 )sin(ϕ n-1 ), calculate a filtered value in an N-dimensional Euclid space to obtain x ={ x 1 , . . . x n } as a final pixel vector.
10. The terminal according to claim 9 , wherein the processor is further configured to:
based on a horizontal and diagonal searching format, or a vertical and horizontal searching format, determine the first pixel set;
obtain the spectral color information for the first pixel set; and
use the spectral color information for the first pixel set to form a vector of each spectral color component of the first pixel.
11. The terminal according to claim 9 , wherein the processor is further configured to:
for the element of each dimension vector x i , calculate a one-dimensional distance
S
i
1
=
∑
j
=
1
N
x
i
-
x
j
1
,
i
=
1
,
…
N
and a two-dimensional distance
S
i
2
=
∑
j
=
1
N
x
i
-
x
j
2
,
i
=
1
,
…
N
between the element and a neighbouring element, and count a sum S i of the one-dimensional distance and the two-dimensional distance as S i =S 1 i +S 2 i ; and
determine the element x i corresponding to a minimum value in the S i as the corresponding vector median.