Assisted photo-tagging with facial recognition models
Embodiments of the invention perform assisted tagging of images, including tagging of people, locations, and activities depicted in those images. A batch of images is received comprising images of faces, including at least some faces that have not yet been tagged. A facial recognition algorithm is applied to the faces to determine matching data comprising possible tags for each untagged face. A logic engine applies logic rules to reduce the likelihood that certain matches are correct. The most likely match from among the possible matches is selected for suggestion to the user for verification. Once verified, the metadata of the image indicating the recognized people within the image is updated.
1. A computer system for tagging images, comprising:
one or more computer processors configured to receive a batch of images comprising images of faces,
to identify a portion of an image in the batch that corresponds to an untagged face,
to apply a facial recognition algorithm to the portion of the image, wherein the facial recognition algorithm takes input from a plurality of facial recognition models, wherein the plurality of facial recognition models each is associated with a known person,
to receive from the facial recognition algorithm, matching data comprising possible tags for the untagged face, wherein each of the possible tags for the untagged face is associated with one of the known persons and a probability factor,
to determine a likelihood for one of the possible tags to correctly match the untagged face based on logic rules, and
to automatically select a provisional tag for the untagged face from the possible tags based on the likelihood,
wherein the one or more computer processors are configured to identify a highest probability factor among probability factors associated with known persons for multiple untagged faces;
to assign a first untagged face to a first known person based on the highest probability factor; and
to eliminate at least one of the possible tags from a second untagged face based on the highest probability factor.
2. The computer system of claim 1 , wherein the one or more computer processors are configured to determine a location at which the image was taken, wherein the logic rules specify that the untagged face in the image cannot be tagged to a person that is known to be at a different location when the image was taken.
3. The computer system of claim 1 , wherein the one or more computer processors are configured to determine an age of the untagged face in the image, wherein the logic rules specify that the untagged face cannot be tagged to a person that has a different age from the age determined for the untagged face.
4. The computer system of claim 1 , wherein the logic rules specify that the untagged face in the image cannot be tagged to a person that is already tagged to another face in the image.
5. The computer system of claim 1 , wherein the one or more computer processors are configured to send a facial recognition request, the request including the portion of the image to a facial recognition sever and to receive the matching data from the facial recognition server.
6. The computer system of claim 1 , wherein the one or more computer processors are configured to determine a gender of the untagged face in the image, wherein the logic rules specify that the untagged face cannot be tagged to a person that has a different gender from the gender determined for the untagged face.
7. The computer system of claim 1 , wherein the matching data further comprises at least one of: an estimated age for the face and a predicted gender for the face.
8. The computer system of claim 1 , wherein the matching data further comprises an unknown person tag indicating that no match was found.
9. The computer system of claim 8 , wherein the one or more computer processors are configured to determine importance of people tagged as unknown based on a frequency that faces of the people appear in a user's images.
10. The computer system of claim 9 , wherein the one or more computer processors are configured to present unknown people for identification by a user in order of importance.
11. The computer system of claim 1 , wherein the one or more computer processors are configured
to send, to a client device, a plurality of provisional tags and the corresponding untagged faces, for display at the client device;
to receive an indication that one or more of the provisional tags are incorrect from the client device;
to replace the one or more provisional tags that are indicated to be incorrect with a provisional unknown person tag; and
to verify tags not indicated to be incorrect.
12. The computer system of claim 11 , wherein the one or more computer processors are configured
to group a subset of faces with an unknown person tag into a group based on a similarity between matching data corresponding to each of the subset; and
to sending, to the client device, a request for a user provide an identity for the group and indicate any faces in the group that do not correspond to the provided identity.
13. The computer system of claim 11 , wherein the one or more computer processors are configured to receive an input from the user that indicates at least one of the untagged faces that has been incorrectly tagged.
14. The computer system of claim 11 , wherein the one or more computer processors are configured to update at least one of facial recognition models based on the tags that are not indicated to be incorrect.
15. The computer system of claim 11 , wherein the one or more computer processors are configured to update at least one of facial recognition models based on the provisional tags indicated to be incorrect.
16. A computer system for tagging images, comprising:
one or more computer processors configured to receive a batch of images,
to compare location metadata for each of the images to a database of known locations, each known location comprising a name and a geographic area,
to tag images with location metadata indicating a location within the geographic area of a known location as corresponding to that known location,
to create a group comprising a plurality of images that do not correspond to a known location, the plurality of images having corresponding location metadata indicating a location within a threshold distance of other images in the group, and
to send a request for a user to provide a name for the location and indicate any images in the group that do not correspond to the named location.
17. The computer system of claim 16 , wherein the one or more computer processors configured to determine location metadata for at least one image based on an IP address of a device used to upload the at least one image.
18. A computer system for tagging images, comprising:
one or more computer processors configured to receive a batch of images,
to compare a timestamp metadata for each of the images to a database of known activities, each known activity comprising at least a name and a time range,
to tag images as corresponding to a known activity responsive to the images having timestamp metadata within the time range of the known activity,
to create a group comprising a plurality of images that do not correspond to a known activity, each image in the group having a timestamp within a threshold time of a timestamp of least one other image in the group, and
to send a request for a user to provide a name for an activity and indicate any images in the group that to not correspond to the activity.
19. The computer system of claim 18 , wherein each image in the group has a location within a threshold distance of a location of another image in the group.
20. The computer system of claim 19 , wherein the one or more computer processors configured to determine location metadata for at least one image in the group based on an IP address of a device used to upload the at least one image in the group.