Method and system for rapid retrieval of target images based on artificial intelligence
A method and system for rapid retrieval of target images based on artificial intelligence, obtaining a template image and a plurality of known labels corresponding to the template image; extracting an image to be detected from a target image database; inputting both the image to be detected and the template image into a trained convolutional neural network, and outputting a hash code of the image to be detected and a hash code of the template image; obtaining a similarity between the images based on a Hamming distance between the hash codes, then selecting one or more images to be detected with the similarity higher than a set threshold as a retrieval result to output. Accordingly, the method and system is able to better cope with the retrieval of items in complex scenarios.
1. A method for rapid retrieval of target images based on artificial intelligence, comprising:
obtaining a template image and a plurality of known labels corresponding to the template image;
extracting an image to be detected from a target image database;
inputting both the image to be detected and the template image into a trained convolutional neural network, and outputting a hash code of the image to be detected and a hash code of the template image; and
obtaining a similarity between the image to be detected and the template image based on a Hamming distance between the hash code of the image to be detected and the hash code of the template image, and the smaller of the Hamming distance indicates the higher of the similarity, then selecting one or more images to be detected with the similarity higher than a set threshold as a retrieval result to output;
based on different situations, different loss functions are used for training, comprising:
when it is required to make prediction scores of all negative samples as low as possible and prediction scores of all positive samples as high as possible, and a range of values of the similarity scores is [−1, 1], then using a unified loss function L u based on the similarity of hash codes;
when the loss function L u is required to focus on positive samples with low similarity prediction scores, then using a loss function L su that weights an interval of the similarity prediction scores of positive and negative samples;
when it is required to make the prediction scores of all negative samples as low as possible and the prediction scores of all positive samples as high as possible, and the range of values of the used similarity scores is [0, 1], then using a loss function L c based on the similarity of hash codes;
when the loss function L c is required to focus on positive samples with low similarity prediction scores, then using a loss function L sc that weights the interval of the similarity prediction scores of the positive and negative samples;
when the loss function L sc is required to expect a significant difference between the similarity prediction scores of the negative samples and the similarity prediction scores of the positive samples, then using a loss function L h ;
when the loss function L su is required to update a similarity matrix in a process of optimization, then using a unified loss function L sus , weighted based on an interval of the optimized similarity matrix;
when the loss function L sc is required to update the similarity matrix in the process of optimization, then using a circular loss function L scs weighted based on the interval of the optimized similarity matrix; and
when the loss function L h is required to update the similarity matrix in the process of optimization, then using a loss function L hs .
2. The method as claimed in claim 1 , wherein the convolutional neural network is an improved convolutional neural network CNN-F (Convolutional Neural Networks and the Improved Fisher Vector Network);
wherein, the improved convolutional neural network CNN-F, a network structure thereof comprises:
sequentially connected a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a first fully connected layer, a second fully connected layer, a third fully connected layer and a Tanh function layer; an output dimension of the third fully connected layer is set to K dimensions.
3. The method as claimed in claim 1 , wherein the trained convolutional neural network, a training step thereof comprises:
constructing a training set and a test set; the training set and the test set, both comprising: images with known labels;
inputting the images with known labels of the training set into the convolutional neural network for training, and then outputting hash codes of the images with known labels by the convolutional neural network; constructing a loss function based on the hash codes of the images with known labels and the known labels; stopping the training when the loss function reaches a minimum value; and
inputting the test set into the convolutional neural network for testing, stopping the testing when an accuracy exceeds the set threshold, and identifying the current convolutional neural network as the trained convolutional neural network.
4. The method as claimed in claim 1 , wherein the trained convolutional neural network is obtained by using the different loss functions for training based on different situations.
5. The method as claimed in claim 1 , wherein the unified loss function L u based on the similarity of the hash codes, a formula thereof is expressed as:
L
u
=
1
n
∑
1
n
log
[
1
+
1
❘
"\[LeftBracketingBar]"
X
i
+
❘
"\[RightBracketingBar]"
❘
"\[LeftBracketingBar]"
X
i
-
❘
"\[RightBracketingBar]"
∑
x
k
∈
X
i
-
exp
(
γ
(
φ
ik
+
m
)
)
∑
x
j
∈
X
i
+
exp
(
γ
(
-
φ
ij
)
)
]
(
1
)
wherein, |g| denotes the number of samples in the set, X i + denotes the set of similar samples of the samples x i , X i − denotes the set of dissimilar samples of the samples x i , γ is a scaling factor, m is an interval, n is the number of images in the training set, φ ik is the prediction sample similarity of the hash codes of the samples x i and x k , φ ij is the prediction sample similarity of the hash codes of the samples x i and x j , wherein, x j is the j-th sample, x k is the k-th sample;
the loss function that weights the interval of the similarity prediction scores of positive and negative samples is expressed as:
L
su
=
1
n
∑
1
n
log
[
1
+
1
❘
"\[LeftBracketingBar]"
X
i
+
❘
"\[RightBracketingBar]"
❘
"\[LeftBracketingBar]"
X
i
-
❘
"\[RightBracketingBar]"
∑
x
k
∈
X
i
-
exp
(
γ
(
φ
ik
)
)
∑
x
j
∈
X
i
+
exp
(
γ
(
-
φ
ij
+
s
ij
·
m
)
)
]
(
2
)
wherein, s ij is the similarity of the sample x i and x j .
6. The method as claimed in claim 1 , wherein the loss function L c based on the similarity of the hash codes is expressed as:
L
c
=
1
n
∑
1
n
log
[
1
+
1
❘
"\[LeftBracketingBar]"
X
i
+
❘
"\[RightBracketingBar]"
❘
"\[LeftBracketingBar]"
X
i
-
❘
"\[RightBracketingBar]"
∑
x
k
∈
X
i
-
exp
(
γ
(
φ
ik
+
1
)
2
)
∑
x
j
∈
X
i
+
exp
(
γ
(
φ
ij
-
1
)
2
-
m
)
]
(
3
)
wherein, L c is a circular loss function based on the similarity of the hash codes;
the loss function L sc that weights the interval of the similarity prediction scores of positive and negative samples is expressed as:
L
sc
=
1
n
∑
1
n
log
[
1
+
1
❘
"\[LeftBracketingBar]"
X
i
+
❘
"\[RightBracketingBar]"
❘
"\[LeftBracketingBar]"
X
i
-
❘
"\[RightBracketingBar]"
∑
x
k
∈
X
i
-
exp
(
γ
(
φ
ik
+
1
)
2
)
∑
x
j
∈
X
i
+
exp
(
γ
(
φ
ij
-
1
)
2
-
(
s
ij
-
1
)
·
m
)
]
(
4
)
wherein, L sc denotes the round loss function that weights the interval of the similarity prediction scores of positive and negative samples.
7. The method as claimed in claim 1 , wherein the loss function L h is expressed as:
L
n
=
1
n
∑
1
n
log
[
1
+
1
❘
"\[LeftBracketingBar]"
X
i
+
❘
"\[RightBracketingBar]"
❘
"\[LeftBracketingBar]"
X
i
-
❘
"\[RightBracketingBar]"
∑
x
k
∈
X
i
-
exp
(
γ
(
max
(
0
,
φ
ik
+
t
)
)
2
)
∑
x
j
∈
X
i
+
exp
(
γ
(
max
(
0
,
φ
ij
-
s
ij
α
)
)
2
-
m
)
]
(
5
)
wherein, L h denotes the circular loss function that combines the characteristics of the hash retrieval task, t is a hyper-parameter that regulates the highest similarity score for negative samples, and α is a hyper-parameter that regulates the lowest similarity score for positive samples.
8. The method as claimed in claim 1 , wherein the unified loss function L sus , weighted based on the interval of the optimized similarity matrix is expressed as:
L
sus
=
1
n
∑
1
n
log
[
1
+
1
❘
"\[LeftBracketingBar]"
X
i
+
❘
"\[RightBracketingBar]"
❘
"\[LeftBracketingBar]"
X
i
-
❘
"\[RightBracketingBar]"
∑
x
k
∈
X
i
-
exp
(
γ
(
φ
ik
)
)
∑
x
j
∈
X
i
+
exp
(
γ
(
-
φ
ij
+
s
ij
′
·
m
)
)
]
;
(
6
)
the circular loss function L scs weighted based on the interval of the optimized similarity matrix is expressed as:
L
scs
=
1
n
∑
1
n
log
[
1
+
1
❘
"\[LeftBracketingBar]"
X
i
+
❘
"\[RightBracketingBar]"
❘
"\[LeftBracketingBar]"
X
i
-
❘
"\[RightBracketingBar]"
∑
x
k
∈
X
i
-
exp
(
γ
(
φ
ik
+
1
)
2
)
∑
x
j
∈
X
i
+
exp
(
γ
(
φ
ij
-
1
)
2
-
(
s
ij
′
-
1
)
·
m
)
]
;
(
7
)
the circular loss function L hs based on the similarity matrix combined with the characteristics of the hash retrieval task is expressed as:
L
ns
=
1
n
∑
1
n
log
[
1
+
1
❘
"\[LeftBracketingBar]"
X
i
+
❘
"\[RightBracketingBar]"
❘
"\[LeftBracketingBar]"
X
i
-
❘
"\[RightBracketingBar]"
∑
x
k
∈
X
i
-
exp
(
γ
(
max
(
0
,
φ
ik
+
t
)
)
2
)
∑
x
j
∈
X
i
+
exp
(
γ
(
max
(
0
,
φ
ij
-
s
ij
α
′
)
)
2
-
m
)
]
.
(
8
)
9. A system for rapid retrieval of target images based on artificial intelligence adopted by a method of claim 1 , comprising:
an acquisition module, configured to: obtaining a template image and a plurality of known labels corresponding to the template image;
an extraction module, configured to: extracting an image to be detected from a target image database;
a conversion module, configured to: inputting both the image to be detected and the template image into a trained convolutional neural network, then outputting a hash code of the image to be detected and a hash code of the template image; and
an output module, configured to: obtaining a similarity between the image to be detected and the template image based on a Hamming distance between the hash code of the image to be detected and the hash code of the template image, and the smaller of the Hamming distance indicates the higher of the similarity, and then selecting one or more images to be detected with the similarity higher than a set threshold as a retrieval result to output.