Automated image retrieval with image graph
An image retrieval system receives an image for which to identify relevant images from an image repository. Relevant images may be of the same environment or object and features and other characteristics. Images in the repository are represented in an image retrieval graph by a set of image nodes connected by edges to other related image nodes with edge weights representing the similarity of the nodes to each other. Based on the received image, the image traversal system identifies an image in the image retrieval graph and alternatively explores and traverses (also termed “exploits”) the image nodes with the edge weights. In the exploration step, image nodes in an exploration set are evaluated to identify connected nodes that are added to a traversal set of image nodes. In the traversal step, the relevant nodes in the traversal set are added to the exploration set and a query result set.
1. A computer system for automated image retrieval, the computer system comprising:
a processor configured to execute instructions;
a non-transient computer-readable medium comprising instructions executable by the processor for:
determining similarity scores for a query image for each other image of a set of other images in the set of images, the similarity scores based on similarity of an image descriptor of the query image and an image descriptor of the other image;
connecting, in an image retrieval graph, a set of image nodes, which correspond to a subset of other images selected based on the similarity scores, to a query node corresponding to the query image with connections having an edge weight based on inliers between the connected nodes; and
identifying relevant images to the query image based on traversal of the image retrieval graph from the query node based on edge weights between nodes in the image retrieval graph.
2. The computer system of claim 1 , wherein identifying relevant images to the query image based on traversal of the image retrieval graph comprises traversal of the image retrieval graph with an explore-exploit algorithm.
3. The computer system of claim 1 , wherein the subset of other images are selected based on a ranked number of a k-nearest neighbor (k-NN) algorithm applied to the similarity scores.
4. The computer system of claim 1 , wherein the instructions are further executable for determining the edge weight for a connection between the query node and another node based on an inlier score of a Random Sample Consensus (RANSAC) algorithm applied to the query image with respect to the other image associated with the other node.
5. The computer system of claim 1 , wherein the image descriptor of the query image comprises a global description of the query image.
6. The computer system of claim 1 , wherein the instructions are further executable for determining the image descriptor of the query image by applying the query image to a computer model that outputs a compact vector representation of the query image.
7. The computer system of claim 1 , wherein the image descriptor includes one or more feature descriptors describing portions of the image.
8. The computer system of claim 1 , wherein determining the similarity scores comprises applying a vector similarity function to the image descriptor of the query image and the image descriptor of the other image.
9. A method for automated image retrieval, the method comprising:
determining similarity scores for a query image for each other image of a set of other images in the set of images, the similarity scores based on similarity of an image descriptor of the query image and an image descriptor of the other image;
connecting, in an image retrieval graph, a set of image nodes, which correspond to a subset of other images selected based on the similarity scores, to a query node corresponding to the query image with connections having an edge weight based on inliers between the connected nodes; and
identifying relevant images to the query image based on traversal of the image retrieval graph from the query node based on edge weights between nodes in the image retrieval graph.
10. The method of claim 9 , wherein identifying relevant images to the query image based on traversal of the image retrieval graph comprises traversal of the image retrieval graph with an explore-exploit algorithm.
11. The method of claim 9 , wherein the subset of other images are selected based on a ranked number of a k-nearest neighbor (k-NN) algorithm applied to the similarity scores.
12. The method of claim 9 , further comprising determining the edge weight for a connection between the query node and another node based on an inlier score of a Random Sample Consensus (RANSAC) algorithm applied to the query image with respect to the other image associated with the other node.
13. The method of claim 9 , wherein the image descriptor of the query image comprises a global description of the query image.
14. The method of claim 9 , further comprising determining the image descriptor of the query image by applying the query image to a computer model that outputs a compact vector representation of the query image.
15. The method of claim 9 , wherein the image descriptor includes one or more feature descriptors describing portions of the image.
16. The method of claim 9 , wherein determining the similarity scores comprises applying a vector similarity function to the image descriptor of the query image and the image descriptor of the other image.
17. A non-transitory computer-readable medium containing computer program code executable by a processor for the processor for:
determining similarity scores for a query image for each other image of a set of other images in the set of images, the similarity scores based on similarity of an image descriptor of the query image and an image descriptor of the other image;
connecting, in an image retrieval graph, a set of image nodes, which correspond to a subset of other images selected based on the similarity scores, to a query node corresponding to the query image with connections having an edge weight based on inliers between the connected nodes; and
identifying relevant images to the query image based on traversal of the image retrieval graph from the query node based on edge weights between nodes in the image retrieval graph.
18. The non-transitory computer-readable medium of claim 17 , wherein identifying relevant images to the query image based on traversal of the image retrieval graph comprises traversal of the image retrieval graph with an explore-exploit algorithm.
19. The non-transitory computer-readable medium of claim 17 , wherein the subset of other images are selected based on a ranked number of a k-nearest neighbor (k-NN) algorithm applied to the similarity scores.
20. The non-transitory computer-readable medium of claim 17 , wherein the computer program code is further executable for determining the edge weight for a connection between the query node and another node based on an inlier score of a Random Sample Consensus (RANSAC) algorithm applied to the query image with respect to the other image associated with the other node.