Query specific fusion for image retrieval
View Patent ↗Systems and methods for image retrieval include constructing a plurality of graphs including a first graph for candidate images retrieved based upon holistic features of a query image and a second graph for candidate images retrieved based upon local features of the query image, wherein constructing includes weighting connected images based upon a Jaccard similarity coefficient. The plurality of graphs are fused to provide a fused graph. Candidate images of the fused graph are ranked, using a processor, to provide retrieval results of the query image.
1. A method for image retrieval, comprising:
capturing images from a camera sensor or from a computer generated display memory;
constructing a plurality of graphs including a first graph for candidate images retrieved based upon holistic features of a query image and a second graph for candidate images retrieved based upon local features of the query image, wherein constructing includes weighting connected images based upon a Jaccard similarity coefficient determined as:
w
(
q
,
d
)
=
α
N
k
(
q
)
⋂
N
k
(
d
)
N
k
(
q
)
⋃
N
k
(
d
)
where w is weight of an edge for images q and d that are reciprocal neighbors, N represents a neighborhood, and α is a decay coefficient related to number of hops to the query;
fusing the plurality of graphs to provide a fused graph; and
ranking, using a processor, candidate images of the fused graph to provide retrieval results of the query image.
2. The method as recited in claim 1 , wherein fusing includes combining weights of the connected images from each of the plurality of graphs.
3. The method as recited in claim 1 , wherein ranking includes determining a maximum weighted subgraph.
4. The method as recited in claim 3 , wherein ranking includes iteratively expanding the maximum weighted subgraph by adding an image to the subgraph with a greatest weight that is connected to the subgraph.
5. The method as recited in claim 4 , wherein ranking includes ranking images according to an order they were added to the subgraph.
6. A method for image retrieval, comprising:
capturing images from a camera sensor or from a computer generated display memory;
constructing a plurality of graphs including a first graph for candidate images retrieved based upon holistic features of a query image and a second graph for candidate images retrieved based upon local features of the query image, wherein constructing includes weighting connected images based upon a Jaccard similarity coefficient determined as:
w
(
q
,
d
)
=
α
N
k
(
q
)
⋂
N
k
(
d
)
N
k
(
q
)
⋃
N
k
(
d
)
where w is weight of an edge for images q and d that are reciprocal neighbors, N represents a neighborhood, and α is a decay coefficient related to number of hops to the query;
fusing the plurality of graphs to provide a fused graph; and
ranking, using a processor, candidate images of the fused graph to provide retrieval results of the query image by determining a maximum weighted subgraph, the subgraph being iteratively expanded by adding an image to the subgraph with a greatest weight that is connected to the subgraph.
7. A system for image retrieval, comprising:
a processor;
a graph construction module configured to construct a plurality of graphs including a first graph for candidate images retrieved based upon holistic features of a query image and a second graph for candidate images retrieved based upon local features of the query image, the graph construction module further configured to weight connected images based upon a Jaccard similarity coefficient determined as:
w
(
q
,
d
)
=
α
N
k
(
q
)
⋂
N
k
(
d
)
N
k
(
q
)
⋃
N
k
(
d
)
where w is weight of an edge for images q and d that are reciprocal neighbors, N represents a neighborhood, and a is a decay coefficient related to number of hops to the query;
a graph fusion module configured to fuse the plurality of graphs to provide a fused graph; and
a link analysis module configured to rank, using a processor, candidate images of the fused graph to provide retrieval results of the query image.
8. The system as recited in claim 7 , wherein the graph fusion module is further configured to combine weights of the connected images from each of the plurality of graphs.
9. The system as recited in claim 7 , wherein the link analysis module is further configured to determine a maximum weighted subgraph.
10. The system as recited in claim 9 , wherein the link analysis module is further configured to iteratively expand the maximum weighted subgraph by adding an image to the subgraph with a greatest weight that is connected to the subgraph.
11. The system as recited in claim 10 , wherein the link analysis module is further configured to rank images according to an order they were added to the subgraph.