IP Library › Granted Patent US 10,607,062
Granted Patent B2
US 10,607,062 · App. 15/692,494 · Granted Mar 31, 2020

Grouping and ranking images based on facial recognition data

Inventors: Chunkit Jacky Chan (Redmond, WA); David Lee (Redmond, WA); Allison Light (Seattle, WA)
Assignee: Microsoft Technology Licensing, LLC
G06K9/00221G06F16/583G06K9/00677G06K9/00288
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,607,062
App. No.
15/692,494
Granted
Mar 31, 2020
Kind
B2
Abstract

Technologies for grouping images, and ranking the images and the groupings, based on entities shown in the images. Images may be grouped based on faces shown in the images. Different images with faces that indicate the same entity (e.g., Adam) may be automatically grouped together. Different images with faces that indicate the same multiple entities (e.g., the people in my family) may also be automatically grouped together. Such automatic grouping may be based on facial recognition technologies. Further, images and groups of images may be automatically ranked based on the faces shown and entities represented. Such rankings may also be influenced by adjacent data that indicates family and friends and the like, and that can be used to identify such entities in the images.

Claims (34)

1. A method performed on a computing device, the method comprising:

receiving, by the computing device for each image in a set of images, facial recognition data that, for each face in each image, comprises a set of facial feature descriptors, a face score that is based on an open/closed state of each face's eyes and mouth and that indicates an overall quality of each face, and a face signature that, across the images in the set, uniquely identifies an entity that each face represents;

ranking, by the computing device based at least on the face signatures, images of the set into a relative ranking; and

outputting, by the computing device, the relative ranking.

2. The method of claim 1 where the ranking is further based on the face scores.

3. The method of claim 1 where a face in each image with a face score below a particular threshold is not included in the ranking.

4. The method of claim 1 where a multi-entity image in the set of images with a larger number of detected faces is ranked higher than a multi-entity image in the set of images with a smaller number of detected faces.

5. The method of claim 1 further comprising:

grouping, based on the face signatures and the face scores, at least a portion of the images in the set of images into a set of groups.

6. The method of claim 1 where an image in the set of images with a face signature indicating a first entity determined to have a larger number of friends or family is ranked higher than an image in the set of images with a face signature indicating a second entity determined to have a smaller number of friends or family.

7. The method of claim 1 where the ranking is further based on an accumulation of ranking information for all faces detected in each image.

8. A computing device comprising:

at least one processor;

memory that is coupled to the at least one processor and that includes computer-executable instructions that, based on execution by the computing device, configure the computing device to perform actions comprising:

receiving, by the computing device for each image in a set of images, facial recognition data that, for each face in each image, comprises a set of facial feature descriptors, a face score that is based on an open/closed state of each face's eyes and mouth and that indicates an overall quality of each face, and a face signature that, across the images in the set, uniquely identifies an entity that each face represents;

ranking, by the computing device based at least on the face signatures, images of the set into a relative ranking; and

outputting, by the computing device, the relative ranking.

9. The computing device of claim 8 where the ranking is further based on the face scores.

10. The computing device of claim 8 where a face in each image with a face score below a particular threshold is not included in the ranking.

11. The computing device of claim 8 where a multi-entity image in the set of images with a larger number of detected faces is ranked higher than a multi-entity image in the set of images with a smaller number of detected faces.

12. The computing device of claim 8 , the actions further comprising:

grouping, based on the face signatures and the face scores, at least a portion of the images in the set of images into a set of groups.

13. The computing device of claim 8 where an image in the set of images with a face signature indicating a first entity determined to have a larger number of friends or family is ranked higher than an image in the set of images with a face signature indicating a second entity determined to have a smaller number of friends or family.

14. The computing device of claim 8 where the ranking is further based on an accumulation of ranking information for all faces detected in each image.

15. At least one non-transitory computer-readable media that includes computer-executable instructions that, based on execution by a computing device, configure the computing device to perform actions comprising:

receiving, by the computing device for each image in a set of images, facial recognition data that, for each face in each image, comprises a set of facial feature descriptors, a face score that is based on an open/closed state of each face's eyes and mouth and that indicates an overall quality of each face, and a face signature that, across the images in the set, uniquely identifies an entity that each face represents;

ranking, by the computing device based at least on the face signatures, images of the set into a relative ranking; and

outputting, by the computing device, the relative ranking.

16. The at least one non-transitory computer-readable media of claim 15 where the ranking is further based on the face scores, or where the ranking is further based on an accumulation of ranking information for all faces detected in each image.

17. The at least one non-transitory computer-readable media of claim 15 where a face in each image with a face score below a particular threshold is not included in the ranking.

18. The at least one non-transitory computer-readable media of claim 15 where a multi-entity image in the set of images with a larger number of detected faces is ranked higher than a multi-entity image in the set of images with a smaller number of detected faces.

19. The at least one non-transitory computer-readable media of claim 15 , the actions further comprising:

grouping, based on the face signatures and the face scores, at least a portion of the images in the set of images into a set of groups.

20. The at least one non-transitory computer-readable media of claim 15 where an image in the set of images with a face signature indicating a first entity determined to have a larger number of friends or family is ranked higher than an image in the set of images with a face signature indicating a second entity determined to have a smaller number of friends or family.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2017
From: CHAN, CHUNKIT JACKY; LEE, DAVID; LIGHT, ALLISON
To: MICROSOFT CORPORATION
Reel/Frame 043580/0733 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2017
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 043580/0757 →
Continuity (2)
Continuation 14264619 · Apr 29, 2014
Related Publication 20170364737A1 · Dec 21, 2017