IP Library Granted Patent US 11,600,113
Granted Patent B2
US 11,600,113 · App. 17/091,066 · Granted Mar 7, 2023

Deep face recognition based on clustering over unlabeled face data

Inventors: Xiang Yu (Mountain View, CA); Manmohan Chandraker (Santa Clara, CA); Kihyuk Sohn (Fremont, CA); Aruni RoyChowdhury (Amherst, MA)
G06V40/173G06K9/6226G06K9/6257G06N3/04G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,600,113
App. No.
17/091,066
Granted
Mar 7, 2023
Kind
B2
Abstract

A computer-implemented method for implementing face recognition includes obtaining a face recognition model trained on labeled face data, separating, using a mixture of probability distributions, a plurality of unlabeled faces corresponding to unlabeled face data into a set of one or more overlapping unlabeled faces that include overlapping identities to those in the labeled face data and a set of one or more disjoint unlabeled faces that include disjoint identities to those in the labeled face data, clustering the one or more disjoint unlabeled faces using a graph convolutional network to generate one or more cluster assignments, generating a clustering uncertainty associated with the one or more cluster assignments, and retraining the face recognition model on the labeled face data and the unlabeled face data to improve face recognition performance by incorporating the clustering uncertainty.

Claims (43)

1. Computer-implemented method for implementing face recognition, comprising:

obtaining a face recognition model trained on labeled face data;

separating, using a mixture of probability distributions, a plurality of unlabeled faces corresponding to unlabeled face data into a set of one or more overlapping unlabeled faces that include overlapping identities to those in the labeled face data and a set of one or more disjoint unlabeled faces that include disjoint identities to those in the labeled face data;

clustering the one or more disjoint unlabeled faces using a graph convolutional network to generate one or more cluster assignments, including assigning a pseudo-label corresponding to a cluster assignment to each of the one or more disjoint unlabeled faces to obtain one or more pseudo-labeled samples;

generating a clustering uncertainty associated with the one or more cluster assignments; and

retraining the face recognition model on the labeled face data and the unlabeled face data to improve face recognition performance by incorporating the clustering uncertainty;

wherein retraining, the face recognition model further includes:

feeding the labeled face data and the one or more pseudo-labeled samples into a neural network; and

attenuating a training loss of the one or more pseudo-labeled samples b using the clustering uncertainty and an output of the neural network to obtain a weighted training loss.

2. The method as recited in claim 1 , wherein the deep neural network is trained on labeled faces in a fully-supervised fashion.

3. The method as recited in claim 1 , wherein the mixture of probability distributions includes a mixture of Weibull distributions.

4. The method as recited in claim 1 , wherein obtaining the estimated clustering uncertainty further includes learning an unsupervised model of a likelihood of incorrect cluster assignments.

5. The method as recited in claim 1 , further comprising implementing the face recognition model to perform a face recognition task.

6. The method as recited in claim 5 , wherein the face recognition model is implemented within a biometric system that provides a user access to a computing system or device based on an analysis of the user's face.

7. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method for implementing face recognition, the method performed by the computer comprising;

obtaining a face recognition model trained on labeled face data;

separating, using a mixture of probability distributions, a plurality of unlabeled faces corresponding to unlabeled face data into a set of one or more overlapping unlabeled faces that include overlapping identities to those in the labeled face data and a set of one or more disjoint unlabeled faces that include disjoint identities to those in the labeled face data;

clustering the one or more disjoint unlabeled faces using a graph convolutional network to generate one or more cluster assignments, including assigning a pseudo-label corresponding to a cluster assignment to each of the one or more disjoint unlabeled faces to obtain one or more pseudo-labeled samples;

generating a clustering uncertainty associated with the one or more cluster assignments; and

retraining the face recognition model on the labeled face data and the unlabeled face data to improve face recognition performance by incorporating the clustering uncertainty;

wherein retraining the face recognition model further includes:

feeding the labeled face data and the one or more pseudo-labeled samples into a neural network; and

attenuating a training loss of the one or more pseudo-labeled samples by using the clustering uncertainty and an output of the neural network to obtain a weighted training loss.

8. The computer program product as recited in claim 7 , wherein the deep neural networks is trained on labeled faces in a fully-supervised fashion.

9. The computer program product as recited in claim 7 , wherein the mixture of probability distributions includes a mixture of Weibull distributions.

10. The computer program product as recited in claim 7 , wherein obtaining the estimated clustering uncertainty further includes learning an unsupervised model of a likelihood of incorrect cluster assignments.

11. The computer program product as recited in claim 7 , wherein the method further includes implementing the face recognition model to perform a face recognition task.

12. The computer program product as recited in claim 11 , wherein the face recognition model is implemented within a biometric system that provides a user access to a computing system or device based on an analysis of the user's face.

13. A system for implementing face recognition, comprising:

a memory device storing program code; and

at least one processor device operatively coupled to the memory device and configured to execute program code stored on the memory device to:

obtain a face recognition model trained on labeled face data;

separate, using a mixture of probability distributions, a plurality of unlabeled faces corresponding to unlabeled face data into a set of one or more overlapping unlabeled faces that include overlapping identities to those in the labeled face data and a set of one or more disjoint unlabeled faces that include disjoint identities to those in the labeled face data;

cluster the one or more disjoint unlabeled faces using a graph convolutional network to generate one or more cluster assignments by assigning a pseudo-label corresponding to a cluster assignment to each of the one or more disjoint unlabeled faces to obtain one or more pseudo-labeled samples;

generate a clustering uncertainty associated with the one or more cluster assignments; and

retrain the face recognition model on the labeled face data and the unlabeled face data to improve face recognition performance by incorporating the clustering uncertainty;

wherein the at least one processor device is further configured to retrain the face recognition model by;

feeding the labeled face data and the one or more pseudo-labeled samples into a neural network; and

attenuating a training loss of the one or more pseudo-labeled samples by using the clustering uncertainty and an output of the neural network to obtain a weighted training loss.

14. The system as recited in claim 13 , wherein the deep neural network trained on labeled faces in a fully-supervised fashion.

15. The system as recited in claim 11 , wherein of probability distributions includes a mixture of Weibull distributions.

16. The system as recited in claim 13 , Wherein the at least one processor device is further configured to generate the clustering uncertainty by learning an unsupervised model of a likelihood of incorrect cluster assignments.

17. The system as recited in claim 13 , Wherein the at least one processor device is further configured to execute program code stored on the memory device to implement the face recognition model to perform a face recognition task, and wherein the face recognition model is implemented within a biometric system that provides a user access to a computing system or device based on an analysis of the user's face.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 18, 2023
From: NEC LABORATORIES AMERICA, INC.
To: NEC CORPORATION
Reel/Frame 062403/0866 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 6, 2020
From: YU, XIANG; CHANDRAKER, MANMOHAN; SOHN, KIHYUK; ROYCHOWDHURY, ARUNI
To: NEC LABORATORIES AMERICA, INC.
Reel/Frame 054294/0350 →
Continuity (2)
Provisional Application 62934617 · Nov 13, 2019
Related Publication 20210142046A1 · May 13, 2021
Cited By (1)
US 12,597,232