Performing image similarity operations using semantic classification
View Patent ↗Image similarity operations are performed in which a seed image is analyzed, and a set of semantic classifications are determined from analyzing the seed image. The set of semantic classifications can include multiple positive semantic classifications. A distance measure is determined that is specific to the set of semantic classifications. The seed image is compared to a collection of images using the distance measure. A set of similar images is determined from comparing the seed image to the collection of images.
1. A computer-implemented method, comprising:
obtaining a query image of an object;
identifying multiple classes of objects for the object;
for each of the identified multiple classes of objects, (i) identifying a distance function associated with the class of objects and (ii) identifying reference images associated with the class of objects;
for each of the identified reference images, determining a similarity score between the reference image and the query image based on the identified distance functions;
selecting one or more of the reference images based on the determined similarity scores; and
providing the selected one or more of the reference images.
2. The method of claim 1 , wherein each distance function is representative of one or more visual features that are particularly relevant for the class of objects associated with the distance function.
3. The method of claim 1 , wherein determining a similarity score between the reference image and the query image based on the identified distance functions, comprises:
aggregating the identified distance functions associated with the classes of objects to determine an aggregated distance function; and
determining the similarity score between the reference image and the query image based on the aggregated distance function.
4. The method of claim 3 , wherein aggregating the identified distance functions associated with the classes of objects to determine an aggregated distance function comprises:
averaging the identified distance functions; and
selecting the averaged identified distance functions as the aggregated distance function.
5. The method of claim 4 , comprising:
for each of the identified multiple classes of objects, determining a confidence score associated with the class of objects based on the query image,
wherein averaging the identified distance functions comprises weighting the identified distance functions associated with the classes of objects based on the determined confidence scores associated with the classes of objects.
6. The method of claim 1 , wherein selecting one or more of the reference images based on the determined similarity scores, comprises:
determining that one or more of the reference images are associated with determined similarity scores that satisfy a similarity score threshold; and
selecting the one or more of the reference images that are associated with determined similarity scores that satisfy the similarity score threshold.
7. The method of claim 1 , wherein determining a similarity score between the reference image and the query image based on the identified distance functions, comprises:
determining a similarity score between the reference image and the query image based on an identified distance function associated with the class of objects associated with both the reference image and the query image.
8. The method of claim 1 , wherein identifying multiple classes of objects for the object comprises:
determining a color histogram for the query image of the object; and
selecting classes of objects that match the color histogram as the identified multiple classes of objects.
9. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
obtaining a query image of an object;
identifying multiple classes of objects for the object;
for each of the identified multiple classes of objects, (i) identifying a distance function associated with the class of objects and (ii) identifying reference images associated with the class of objects;
for each of the identified reference images, determining a similarity score between the reference image and the query image based on the identified distance functions;
selecting one or more of the reference images based on the determined similarity scores; and
providing the selected one or more of the reference images.
10. The system of claim 9 , wherein each distance function is representative of one or more visual features that are particularly relevant for the class of objects associated with the distance function.
11. The system of claim 9 , wherein determining a similarity score between the reference image and the query image based on the identified distance functions, comprises:
aggregating the identified distance functions associated with the classes of objects to determine an aggregated distance function; and
determining the similarity score between the reference image and the query image based on the aggregated distance function.
12. The system of claim 11 , wherein aggregating the identified distance functions associated with the classes of objects to determine an aggregated distance function comprises:
averaging the identified distance functions; and
selecting the averaged identified distance functions as the aggregated distance function.
13. The system of claim 12 , the operations comprising:
for each of the identified multiple classes of objects, determining a confidence score associated with the class of objects based on the query image,
wherein averaging the identified distance functions comprises weighting the identified distance functions associated with the classes of objects based on the determined confidence scores associated with the classes of objects.
14. The system of claim 9 , wherein selecting one or more of the reference images based on the determined similarity scores, comprises:
determining that one or more of the reference images are associated with determined similarity scores that satisfy a similarity score threshold; and
selecting the one or more of the reference images that are associated with determined similarity scores that satisfy the similarity score threshold.
15. The system of claim 9 , wherein determining a similarity score between the reference image and the query image based on the identified distance functions, comprises:
determining a similarity score between the reference image and the query image based on an identified distance function associated with the class of objects associated with both the reference image and the query image.
16. The system of claim 9 , wherein identifying multiple classes of objects for the object comprises:
determining a color histogram for the query image of the object; and
selecting classes of objects that match the color histogram as the identified multiple classes of objects.
17. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
obtaining a query image of an object;
identifying multiple classes of objects for the object;
for each of the identified multiple classes of objects, (i) identifying a distance function associated with the class of objects and (ii) identifying reference images associated with the class of objects;
for each of the identified reference images, determining a similarity score between the reference image and the query image based on the identified distance functions;
selecting one or more of the reference images based on the determined similarity scores; and
providing the selected one or more of the reference images.
18. The medium of claim 17 , wherein each distance function is representative of one or more visual features that are particularly relevant for the class of objects associated with the distance function.
19. The medium of claim 17 , wherein determining a similarity score between the reference image and the query image based on the identified distance functions, comprises:
aggregating the identified distance functions associated with the classes of objects to determine an aggregated distance function; and
determining the similarity score between the reference image and the query image based on the aggregated distance function.
20. The medium of claim 17 , wherein aggregating the identified distance functions associated with the classes of objects to determine an aggregated distance function comprises:
averaging the identified distance functions; and
selecting the averaged identified distance functions as the aggregated distance function.