Dataset-aware and invariant learning for face recognition
Introduced here is an approach to training a model to perform facial recognition with a designed dataset-aware loss. This approach is particularly useful for large-scale, multi-dataset training since there are no prerequisites for label cleaning. Dataset-aware loss may be built upon the softmax function with a binary dataset indicator. Such an approach allows the dataset-aware loss to be readily combined with other losses to improve performance.
1 . A system for training a model to perform facial recognition, the system comprising:
an embedding network that takes, as input, an image of a face to be recognized and a first network parameter and then outputs an embedding;
a classification network that takes, as input, the embedding and a second network parameter and then outputs a first label that is representative of a proposed identity of an individual to whom the face belongs; and
a dataset classifier network that takes, as input, the embedding and a third network parameter and then outputs a second label that is representative of a mapping of the first label to a dataset to which the image belongs,
wherein at least one of the first network parameter, the second network parameter, or the third network parameter is optimized in a multi-stage process to ensure that embeddings are dataset invariant,
wherein, in a first stage of the multi-stage process, the second and third network parameters are initialized and then the classification network and the dataset classifier network are trained separately, and
wherein, in a second stage of the multi-stage process, the second and third network parameters are tuned following convergence of the classification network and the dataset classifier network.
2 . The system of claim 1 , wherein the embedding is a vector representation of facial features that are extracted from the image by the embedding network.
3 . The system of claim 1 , wherein the first and second network parameters are learned in training through analysis of embeddings, output by the embedding network for facial images that serve as training data, that minimize classification loss.
4 . The system of claim 1 ,
wherein the classification network is further configured to determine a dataset-aware classification loss for the embedding, and
wherein the dataset classifier network is further configured to determine a classification loss for the embedding.
5 . The system of claim 4 , further comprising:
a backpropagation algorithm that, when executed,
backpropagates the dataset-aware classification loss through the classification network, and
backpropagates the classification loss through the dataset classifier network.
6 . The system of claim 5 , wherein the backpropagation algorithm, when executed,
backpropagates, after backpropagating the dataset-aware classification loss through the classification network, the dataset-aware classification loss through the embedding network while modifying the embedding network to train the embedding network; and
backpropagates, after backpropagating the classification loss through the dataset classifier network, the classification loss through the embedding network while modifying the embedding network to train the embedding network.
7 . The system of claim 1 , wherein the embedding network further takes, as input, multiple datasets, and wherein the multiple datasets include (i) a first set of facial images that includes the image and (ii) a second set of facial images that includes a second image of the face of the individual to whom the face belongs.
8 . The system of claim 7 , wherein the image and the second image are different facial images.
9 . The system of claim 7 , wherein each dataset from the multiple datasets is obtained from a different source.
10 . A method comprising:
receiving an image of a face to be recognized;
providing the image, as input, to an embedding network that produces an embedding of features of the face as output;
providing the embedding, as input, to a classification network that produces, as output, a first label that is representative of a proposed identity of an individual to whom the face belongs and a dataset-aware classification loss for the embedding; and
providing the embedding, as input, to a dataset classifier network that produces, as output, a second label that is representative of a mapping of the first label to a dataset to which the image belongs and a classification loss for the embedding;
backpropagating the dataset-aware classification loss through the classification network;
after backpropagating the dataset-aware classification loss through the classification network, backpropagating the dataset-aware classification loss through the embedding network while modifying the embedding network to train the embedding network;
backpropagating the classification loss through the dataset classifier network; and
after backpropagating the classification loss through the dataset classifier network, backpropagating the classification loss through the embedding network while modifying the embedding network to train the embedding network.
11 . The method of claim 10 , wherein the embedding is a vector representation of facial features that are extracted from the image by the embedding network.
12 . The method of claim 10 , further comprising:
providing a first network parameter, as input, to the embedding network;
providing a second network parameter, as input, to the classification network; and
providing a third network parameter, as input, to the dataset classifier network.
13 . The method of claim 12 , wherein the first and second network parameters are learned in training through analysis of embeddings, output by the embedding network for facial images that serve as training data, that minimize classification loss.
14 . The method of claim 12 , further comprising:
optimizing at least one of the first network parameter, the second network parameter, or the third network parameter in a multi-stage process to ensure that embeddings are dataset invariant.
15 . The method of claim 14 , further comprising optimizing at least one of the first network parameter, the second network parameter, or the third network parameter by:
in a first stage of the multi-stage process, initializing the second and third network parameters and then separately training the classification network and the dataset classifier network, and
in a second stage of the multi-stage process, tuning the second and third network parameters following convergence of the classification network and the dataset classifier network.