Automatic discovery of popular landmarks
View Patent ↗In one embodiment the present invention is a method for populating and updating a database of images of landmarks including geo-clustering geo-tagged images according to geographic proximity to generate one or more geo-clusters, and visual-clustering the one or more geo-clusters according to image similarity to generate one or more visual clusters. In another embodiment, the present invention is a system for identifying landmarks from digital images, including the following components: a database of geo-tagged images; a landmark database; a geo-clustering module; and a visual clustering module. In other embodiments the present invention may be a method of enhancing user queries to retrieve images of landmarks, or a method of automatically tagging a new digital image with text labels.
1. A computer-implemented method comprising:
receiving, by one or more processors, a user query;
identifying one or more trigger words in the user query;
selecting one or more tags from a landmark database, the tags corresponding to the one or more trigger words;
supplementing the user query with the one or more tags to generate a supplemented user query that describe a landmark;
in response to receiving the supplemented user query, identifying a plurality of visual clusters from the landmark database wherein the plurality of visual clusters are associated with a landmark based on the supplemented user query;
causing a user interface to be displayed, wherein the user interface includes the plurality of visual clusters;
receiving user input wherein the user input indicates that a first visual cluster of the plurality of visual clusters and a second visual cluster of the plurality of visual clusters are to be merged, wherein the second visual cluster is different than the first visual cluster; and
in response to receiving the user input, updating the landmark database to merge the first visual cluster and the second visual cluster.
2. The method of claim 1 , wherein the user input further indicates that a third visual cluster of the plurality of visual clusters is to be disassociated from the landmark; and wherein updating the landmark database comprises disassociating the third visual cluster from the landmark in the landmark database.
3. The method of claim 1 , wherein the user interface includes a user input graphic enabled to receive the user input.
4. The method of claim 1 , wherein the user interface includes a plurality of user input graphics, wherein each of the plurality of user input graphics is associated with a respective visual cluster of the plurality of visual clusters, and is configured to receive the user input.
5. The method of claim 1 , wherein the user interface includes a plurality of landmarks and one or more corresponding visual clusters.
6. The method of claim 1 , wherein the user interface displays descriptive information comprising:
a number of images;
a popularity of the landmark wherein the popularity is based on a number of one or more authors that have contributed images to the plurality of visual clusters;
an indication that one or more of the plurality of visual clusters have been modified by a user; and
an indication that one or more of the plurality of visual clusters have been verified by the user.
7. The method of claim 1 , further comprising:
wherein the user input further indicates that one or more landmarks associated with the plurality of visual clusters are to be merged.
8. A system comprising:
one or more processors; and
logic encoded in one or more tangible media for execution by the one or more processors and when executed operable to perform operations comprising:
receiving a user query;
identifying one or more trigger words in the user query;
selecting one or more tags from a landmark database, the tags corresponding to the one or more trigger words;
supplementing the user query with the one or more tags to generate a supplemented user query that describe a landmark;
in response to receiving the supplemented user query, identifying a plurality of visual clusters from the landmark database wherein the plurality of visual clusters are associated with a landmark based on the supplemented user query;
causing a user interface to be displayed, wherein the user interface includes the plurality of visual clusters;
receiving user input wherein the user input indicates that a first visual cluster of the plurality of visual clusters and a second visual cluster of the plurality of visual clusters to be merged, wherein the second visual cluster is different than the first visual cluster; and
in response to receiving the user input, updating the landmark database to merge the first visual cluster and the second visual cluster.
9. The system of claim 8 , wherein the user input further indicates that a third visual cluster of the plurality of visual clusters is to be disassociated from the landmark; and wherein updating the landmark database comprises disassociating the third visual cluster from the landmark in the landmark database.
10. The system of claim 8 , wherein the user interface includes a user input graphic enabled to receive the user input.
11. The system of claim 8 , wherein the user interface includes a plurality of user input graphics, wherein each of the plurality of user input graphics is associated with a respective visual cluster of the plurality of visual clusters, and is configured to receive the user input.
12. The system of claim 8 , wherein the user interface includes a plurality of landmarks and one or more corresponding visual clusters.
13. The system of claim 8 , further comprising applying one or more tags to the updated landmark database.
14. The system of claim 8 , further comprising:
wherein the user input further indicates that one or more landmarks associated with the plurality of visual clusters are to be merged.
15. A non-transitory computer readable medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:
receiving a user query;
identifying one or more trigger words in the user query;
selecting one or more tags from a landmark database, the tags corresponding to the one or more trigger words;
supplementing the user query with the one or more tags to generate a supplemented user query that describe a landmark;
in response to receiving the supplemented user query, identifying a plurality of visual clusters from the landmark database wherein the plurality of visual clusters are associated with a landmark based on the supplemented user query;
causing a user interface to be displayed, wherein the user interface includes the plurality of visual clusters;
receiving user input wherein the user input indicates that a first visual cluster of the plurality of visual clusters and a second visual cluster of the plurality of visual clusters are to be merged; and
in response to receiving the user input, updating the landmark database to merge the first visual cluster and the second visual cluster.
16. The non-transitory computer readable medium of claim 15 , wherein the user input further indicates that a third visual cluster of the plurality of visual clusters is to be disassociated from the landmark; and wherein updating the landmark database comprises disassociating the third visual cluster from the landmark in the landmark database.
17. The non-transitory computer readable medium of claim 15 , wherein the user interface includes a user input graphic enabled to receive the user input.
18. The non-transitory computer readable medium of claim 15 , wherein the user interface includes a plurality of user input graphics, wherein each of the plurality of user input graphics is associated with a respective visual cluster of the plurality of visual clusters, and is configured to receive the user input.
19. The non-transitory computer readable medium of claim 15 , wherein the user interface includes a plurality of landmarks and one or more corresponding visual clusters.
20. The non-transitory computer readable medium of claim 15 , wherein the user interface displays descriptive information comprising:
a number of images;
a popularity of the landmark wherein the popularity is based on a number of one or more authors that have contributed images to the plurality of visual clusters;
an indication that one or more of the plurality of visual clusters have been modified by a user; and
an indication that one or more of the plurality of visual clusters have been verified by the user.
21. The non-transitory computer readable medium of claim 15 , further comprising:
receiving text input from one or more users wherein the text input includes one or more new text labels to be assigned to a merged visual cluster of the updated landmark database formed by merging the first visual cluster and the second visual cluster.