Apparatus and method for performing visual search
A method comprises computing a color factor value indicating an amount of color gradients in at least one color channel from the query image. The method comprises combining the color-keypoints with the gray-keypoints when the color factor value is greater than a threshold. A method for performing a visual search comprises extracting a plurality of local descriptors from a query image and then selecting a subset of them based on various criteria's such as visual meaning score. A method comprises aggregating each mean vector for each visual codeword from distances between each visual codeword and local descriptors. The method comprises aggregating variance vector for each visual codeword from the distance between each visual codeword, and local descriptors. The method comprises transmitting aggregated mean vector information and aggregated variance vector information to a search server for efficient image retrieval.
1. A method for performing a visual search, the method comprising,
extracting at least one gray-keypoint from a grayscale of a query image;
computing a color factor value indicating a value of color gradient in at least one color channel from the query image by a client device;
extracting at least one color-keypoint from the at least one color channel when the color factor value is greater than a predetermined threshold;
determining whether the at least one color-keypoint is significantly distinctive from a respective adjacent gray-keypoint than a threshold by the client device; and
combining the least one color-keypoint with the at least one gray-keypoint when the at least one color-keypoint is significantly distinctive from the respective adjacent gray-keypoint than the threshold by the client device.
2. The method of claim 1 , further comprising closing a switch to activate the extracting the at least one color-keypoint from the at least one color channel when the color factor value is greater than a predetermined threshold.
3. The method of claim 1 , wherein the combining of the at least one color-keypoint comprises:
including the extracted at least one gray-keypoint into a pool;
selecting one gray-keypoint with a minimum spatial distance from each color-keypoint, out of the at least one gray-keypoint of the pool;
computing a respective difference of a spatial distance, orientation, and scale between the selected one gray-keypoint and the at least one color-keypoint; and
including the at least one color-keypoint into the pool when one of the respective difference is greater than a corresponding threshold value.
4. The method of claim 3 , further comprising:
replacing the selected one gray-keypoint with the at least one color-keypoint when a peak value of the at least one color-keypoint is greater than that of the selected one gray-keypoint.
5. The method of claim 1 , wherein the color factor value is calculated as follows:
C
=
∑
G
(
x
,
y
)
-
∑
G
i
(
x
,
y
)
n
/
(
h
*
w
)
where G i is a gradient image computed from the i th channel, n is a number of the at least one color channel, h and w denote the height and width of the image, and the sum is over (x,y), and a grayscale image I is extracted from each of the at least one color channel is computed using Sobel kernel K as follows:
G x =K′*I
G y =K*I
G =√{square root over ( G x 2 +G y 2 )}
where the operator ‘*’ represents image convolution G x and G y , wherein K′ is a transposed matrix of K.
6. The method of claim 5 , wherein the at least one color channel includes red, green, and blue channels, and the color factor value is calculated as follows:
C
=
∑
G
(
x
,
y
)
-
G
r
(
x
,
y
)
+
G
g
(
x
,
y
)
+
G
b
(
x
,
y
)
3
/
(
h
*
w
)
where G is a gradient image computed using the grayscale image I, G r is the gradient image obtained from the red channel, G g is obtained from the green channel, and G b is obtained from the blue channel.
7. The method of claim 1 , wherein the color factor value is appended to a global descriptor transmitted to a search server.
8. An apparatus for performing a visual search, the apparatus comprising:
a controller configured to:
extract at least one gray-keypoint from a grayscale of a query image;
compute a color factor value indicating an amount of color gradient in at least one color channel from the query image;
extract at least one color-keypoint from the at least one color channel when the color factor value is greater than a predetermined threshold;
determine whether the at least one color-keypoint is significantly distinctive from a respective adjacent gray-keypoint than a threshold; and
combine the at least one color-keypoint with the at least one gray-keypoint when the at least one color-keypoint is significantly distinctive from the respective adjacent gray-keypoint than the threshold, and
an interface configured to communicate with a server.
9. The apparatus of claim 8 , further comprising a switch configured to activate extracting the at least one color-keypoint from the at least one color channel when the color factor value is greater than a predetermined threshold.
10. The apparatus of claim 8 , wherein the controller is configured to:
replace one keypoint with the color-keypoint when a peak value of the color-keypoint is greater than that of the one keypoint.
11. The apparatus of claim 8 , wherein the controller is further configured to:
include the extracted at least one gray-keypoint into a pool;
select one gray-keypoint with a minimum spatial distance from the at least one color-keypoint, out of the at least one gray-keypoint of the pool;
compute a respective difference of a spatial distance, orientation, and scale between the selected one gray-keypoint and the at least one color-keypoint; and
include the at least one color-keypoint into the pool when one of the respective difference is greater than a corresponding threshold value.
12. The apparatus of claim 11 , wherein the controller is further configured to:
replace the selected one gray-keypoint with the at least one color-keypoint when a peak value of the at least one color-keypoint is greater than that of the selected one gray-keypoint.
13. The apparatus of claim 8 , wherein the color factor value is calculated as follows:
C
=
∑
G
(
x
,
y
)
-
∑
G
i
(
x
,
y
)
n
/
(
h
*
w
)
where G i is a gradient image computed from the i th channel, n is a number of the at least one color channel, h and w denote the height and width of the image, and the sum is over (x,y), and a grayscale image I is extracted from each of the at least one color channel is computed using Sobel kernel K as follows:
G x =K′*I
G y =K*I
G =√{square root over ( G x 2 +G y 2 )}
where the operator ‘*’ represents image convolution of G x and G y , wherein K′ is a transposed matrix of K.
14. The apparatus of claim 13 , wherein the at least one color channel includes red, green, and blue channels, and the color factor value is calculated as follows:
C
=
∑
G
(
x
,
y
)
-
G
r
(
x
,
y
)
+
G
g
(
x
,
y
)
+
G
b
(
x
,
y
)
3
/
(
h
*
w
)
where G is a gradient image computed using the grayscale image I, G r is the gradient image obtained from the red channel, G g is obtained from the green channel, and G b is obtained from the blue channel.
15. The apparatus of claim 8 , wherein the color factor value is appended to a global descriptor transmitted to the server.