Face-aware person re-identification system

1. A method for employing facial information in unsupervised person re-identification, the method comprising:

extracting, by a body feature extractor, body features from a first data stream;

extracting, by a head feature extractor, head features from a second data stream;

outputting a body descriptor vector from the body feature extractor;

outputting a head descriptor vector from the head feature extractor;

concatenating the body descriptor vector and the head descriptor vector to enable a model to generate a descriptor vector;

enhancing the first data stream using the head features and the second data stream using the body features by utilizing cross-task consistency as a self-supervision during training;

training the head feature extractor using a face knowledge distillation loss being applied randomly at each iteration with a particular probability;

employing a two-stream network where each stream processes body and head images separately, and utilizing both body and head appearances together;

preserving a relation between images across two modalities using cross-modal consistency loss;

employing additional face recognition loss for training the face stream of the model using pseudo labels for images from a target domain when frontal faces are visible; and

distilling knowledge obtained from a face recognition engine on the target domain to a head sub-network of the model.

2. The method of claim 1 , wherein the model is trained by using face-based pseudo labels, body-based pseudo labels, and head-based pseudo labels.

3. The method of claim 1 , wherein a face-based pseudo label loss is used to train the head feature extractor.

4. The method of claim 1 , wherein a cross-modal consistency loss is employed as a self-supervision during training.

5. The method of claim 1 , wherein a cross-modal consistency loss is employed in tandem with a face knowledge distillation loss.

6. The method of claim 1 , wherein the head feature extractor crops images from the first data stream around a head region and extracts head features by only using cropped regions.

7. The method of claim 1 , wherein temporal weight averaging is used to average network parameters over iterations to extract features for generating pseudo labels.

8. A non-transitory computer-readable storage medium comprising a computer-readable program for employing facial information in unsupervised person re-identification wherein the computer-readable program when executed on a computer causes the computer to perform steps of:

extracting, by a body feature extractor, body features from a first data stream;

extracting, by a head feature extractor, head features from a second data stream;

outputting a body descriptor vector from the body feature extractor;

outputting a head descriptor vector from the head feature extractor;

concatenating the body descriptor vector and the head descriptor vector to enable a model to generate a descriptor vector;

enhancing the first data stream using the head features and the second data stream using the body features by utilizing cross-task consistency as a self-supervision during training;

training the head feature extractor using a face knowledge distillation loss being applied randomly at each iteration with a particular probability;

employing a two-stream network where each stream processes body and head images separately, and utilizing both body and head appearances together;

preserving a relation between images across two modalities using cross-modal consistency loss;

employing additional face recognition loss for training the face stream of the model using pseudo labels for images from a target domain when frontal faces are visible; and

distilling knowledge obtained from a face recognition engine on the target domain to a head sub-network of the model.

9. The non-transitory computer-readable storage medium of claim 8 , wherein the model is trained by using face-based pseudo labels, body-based pseudo labels, and head-based pseudo labels.

10. The non-transitory computer-readable storage medium of claim 8 , wherein a face-based pseudo label loss is used to train the head feature extractor.

11. The non-transitory computer-readable storage medium of claim 8 , wherein a cross-modal consistency loss is employed as a self-supervision during training.

12. The non-transitory computer-readable storage medium of claim 8 , wherein a cross-modal consistency loss is employed in tandem with a face knowledge distillation loss.

13. The non-transitory computer-readable storage medium of claim 8 , wherein the head feature extractor crops images from the first data stream around a head region and extracts head features by only using cropped regions.

14. The non-transitory computer-readable storage medium of claim 8 , wherein temporal weight averaging is used to average network parameters over iterations to extract features for generating pseudo labels.

15. A system for employing facial information in unsupervised person re-identification, the system comprising:

a memory; and

one or more processors in communication with the memory configured to:

extract, by a body feature extractor, body features from a first data stream;

extract, by a head feature extractor, head features from a second data stream;

output a body descriptor vector from the body feature extractor;

output a head descriptor vector from the head feature extractor;

concatenate the body descriptor vector and the head descriptor vector to enable a model to generate a descriptor vector;

enhance the first data stream using the head features and the second data stream using the body features by utilizing cross-task consistency as a self-supervision during training;

train the head feature extractor using a face knowledge distillation loss being applied randomly at each iteration with a particular probability;

employ a two-stream network where each stream processes body and head images separately, and utilizing both body and head appearances together;

preserve a relation between images across two modalities using cross-modal consistency loss;

employ additional face recognition loss for training the face stream of the model using pseudo labels for images from a target domain when frontal faces are visible; and

distill knowledge obtained from a face recognition engine on the target domain to a head sub-network of the model.

16. The system of claim 15 , wherein the model is trained by using face-based pseudo labels, body-based pseudo labels, and head-based pseudo labels.

17. The system of claim 15 , wherein a face-based pseudo label loss is used to train the head feature extractor.

18. The system of claim 15 , wherein a cross-modal consistency loss is employed as a self-supervision during training.

19. The system of claim 15 , wherein a cross-modal consistency loss is employed in tandem with a face knowledge distillation loss.

20. The system of claim 15 , wherein the head feature extractor crops images from the first data stream around a head region and extracts head features by only using cropped regions.