Corresponding image processing method for compensating colour
A corresponding image processing method is provided. The corresponding image processing method includes a first step for searching for a corresponding point of a corresponding image; a second step for performing radiometric calibration such that a camera response function of the corresponding image has a linear function; a third step for compensating for a color of the corresponding image by calculating a color conversion matrix; and a fourth step for measuring similarity of the corresponding image.
1. A corresponding image processing method comprising:
a first step for searching for a corresponding point between a target image and a corresponding image;
a second step for performing radiometric calibration such that a camera response function of the corresponding image has a linear function;
a third step for compensating for a color of the corresponding image by calculating a color conversion matrix; and
a fourth step for measuring similarity between the target image and the corresponding image,
wherein the first step comprises:
detecting features using a feature detecting method based on a corner point;
deriving first matching points N by matching the features;
deriving second matching points M by removing outlier in the first matching points N; and
determining suitability of the feature detection, and
wherein the determining the suitability of the feature detection comprises:
when a number of the second matching points M is divided by a number of the first matching points N and a quotient falls below 0.7, re-detecting the features using a feature detecting method based on invariant features; and
matching the features and re-deriving second matching points M 2 by removing outlier.
2. The corresponding image processing method of claim 1 , wherein the first step further comprises:
determining corresponding point determination suitability, and
the determining of the corresponding point determination suitability determines whether R, G, B color distribution value of the second matching points M or the second matching points M 2 exceeds or falls below 20% of R, G, B color distribution value of the entire image.
3. The corresponding image processing method of claim 2 , wherein the first step further comprises:
when the R, G, B color distribution value of the second matching points M or the second matching points M 2 exceeds 20% of the R, G, B color distribution value of the entire image, determining the second matching point M or M 2 as a corresponding point;
when the R, G, B color distribution value of the second matching point M or M 2 falls below 20% of the R, G, B color distribution value of the entire image, re-deriving a matching point using dense stereo matching; and
determining the re-derived matching point as the corresponding point.
4. The corresponding image processing method of claim 1 , wherein the second step comprises:
calculating the response function of a camera;
calculating an inverse function of the response function; and
converting an image to an image having a linear camera response function by applying the calculated inverse function to the image.
5. The corresponding image processing method of claim 1 , wherein the third step is any one of a method for calculating the color conversion matrix based on least square based on Equation 1, a method for calculating a 3×3 RGB color conversion matrix based on Equation 2, and a method for calculating a color conversion matrix comprising a nonlinear polynomial based on Equation 3:
∑
x
=
1
NS
(
Ic
s
⟶
-
(
a
c
Tc
s
⟶
+
b
c
)
)
2
,
c
∈
R
,
G
,
B
(
1
)
where Ic s denotes an RGB color of a target image, Tc s denotes an RGB color of an input image to convert, and a c and b c denote coefficients for matching the two colors,
∑
s
=
1
NS
(
I
s
⟶
-
T
RGB
·
T
s
⟶
)
2
(
2
)
where I s denotes R, G, B vector of an S-th corresponding point in the input image, T RGB denotes each element of the color conversion matrix, and T s denotes the R, G, B vector of the S-th corresponding point in the target image, and
∑
k
=
1
D
(
t
rck
Ir
s
k
+
t
gck
Ig
s
k
+
t
bck
Ib
s
k
)
+
t
c
0
≅
Tc
s
(
3
)
where D denotes a degree in nonlinear conversion, t rck , t gck , and t bck denote coefficients corresponding to the respective colors of each degree, t c0 , denotes a coefficient in the degree 0, Ir s k , Ig s k , and Ib s k denote R, G, B values of each degree of the input image, and Tc s denotes one channel value (one of R, G, and B) of the target image.
6. The corresponding image processing method of claim 1 , wherein the fourth step comprises:
a 4-1 step for searching for a plurality of corresponding points between corresponding images;
a 4-2 step for calculating a first similarity indicating similarity level of the corresponding blocks by generating a plurality of corresponding blocks comprising the corresponding points in the corresponding image and comparing color distribution of the corresponding blocks;
a 4-3 step for calculating a second similarity indicating similarity level of the overlapping regions by generating overlap regions where images are overlapped in a whole region of the corresponding image and comparing color distribution of the overlap regions; and
a 4-4 step for measuring similarity of the corresponding image by applying a weight to the first similarity and the second similarity.
7. The corresponding image processing method of claim 6 , wherein the 4-2 step comprises:
generating the plurality of the corresponding blocks comprising the corresponding points in the corresponding image;
calculating and comparing a normalized average of R, G, B color intensity value of the corresponding blocks;
calculating and comparing a histogram of the corresponding blocks; and calculating the first similarity by applying a weight to the normalized average difference and the histogram difference.
8. The corresponding image processing method of claim 7 , wherein the calculating of the first similarity is based on Equation 4:
S
block
(
i
,
j
)
=
∑
c
=
r
,
g
,
b
S
μ
c
(
i
,
j
)
+
α
1
·
∑
c
=
r
,
g
,
b
∑
k
=
1
N
[
d
k
c
(
H
i
,
H
j
)
]
2
(
4
)
where S block (i,j) denotes the first similarity, S μ c (i,j) denotes the difference of the normalized average of the R, G, B color intensity value in a channel C of the corresponding block, d k c (H i ,H j ) denotes the histogram difference of the corresponding image of the channel C, the channel C indicates one channel selected from R, G, and B, and α 1 is the weight.
9. The corresponding image processing method of claim 6 , wherein the 4-3 step comprises:
generating the overlap regions where the images are overlapped in the whole region of the corresponding image;
calculating and comparing the normalized average of the R, G, B color intensity value of the overlapping regions;
calculating and comparing the histogram of the overlap regions; and
calculating the second similarity by applying a weight to the normalized average difference and the histogram difference.
10. The corresponding image processing method of claim 9 , wherein the calculating of the second similarity is based on Equation 5:
S
image
(
i
,
j
)
=
∑
c
=
R
,
G
,
B
μ
i
c
σ
i
c
-
μ
j
c
σ
j
c
+
α
2
·
∑
c
=
R
,
G
,
B
d
c
(
H
i
,
H
j
)
(
5
)
where S image (i,j) denotes the second similarity, μ i c and μ j c denote the average of the R, G, B color intensity value in the channel C in an image i or j, σ i c of and σ j c denote a standard deviation of the R, G, B color intensity value in the channel C in the image i or j, d c (H i ,H j ) denotes the histogram difference of the corresponding image in the channel C, the channel C indicates one selected from R, G, and B, and α 2 is the weight.
11. The corresponding image processing method of claim 6 , wherein the 4-4 step is based on Equation 6 and Equation 7:
S total ( i,j )= S block ( i,j )+β· S image ( i,j ) (6)
where S block (i,j) denotes the first similarity, S image (i,j) denotes the second similarity, and β denotes the weight, and
β
=
0.1
+
0.4
×
(
1
-
⋃
k
=
1
N
Region
i
(
k
)
Overlap
Region
i
)
+
0.4
×
(
1
-
⋃
k
=
1
N
Region
j
(
k
)
Overlap
Region
j
)
(
7
)
where Overlap Region i , denotes the overlap region in the image i, Overlap Region j , denotes the overlap region in the image j, Region i (k) denotes a k-th block region in the image i, and Region j (k) denotes a k-th block region in the image j.
12. A non-transitory computer-readable recording medium containing a program for executing the method of claim 1 in a computer.