Compact, clustering-based indexes for large-scale real-time lookups on streaming videos
View Patent ↗Systems and methods for recognizing a face are disclosed and includes receiving images of faces; generating feature vectors of the images; generating clusters of feature vectors each with a centroids or a cluster representative; for a query to search for a face, generating corresponding feature vectors for the face and comparing the feature vector with the centroids of all clusters; for clusters above a similarity threshold, comparing cluster members with the corresponding feature vector; and indicating as matching candidates for cluster members with similarity above a threshold.
1. A method for recognizing a face, comprising:
receiving images of training faces;
generating feature vectors of the images;
generating clusters from the feature vectors each with one or more centroids or a cluster representative;
for a query to search for a query face, generating query feature vectors for the query face and comparing the query feature vectors with the centroids of all clusters to find one or more similar clusters;
for clusters above a similarity threshold, comparing feature vectors of corresponding members of the clusters with the query feature vectors; and
indicating as matching candidates for cluster members with similarity above a threshold, wherein each cluster model size is sub-linear or logarithmic in the number of the training faces (or features) in a database.
2. The method of claim 1 , comprising applying a clustering-based index for features extracted from real-time video streams.
3. The method of claim 1 , comprising clustering the training faces into a pre-determined number of clusters with an unsupervised clustering method.
4. The method of claim 1 , comprising clustering the training faces into a pre-determined number of clusters with k-medoids.
5. The method of claim 1 , wherein each image of the training faces is assigned to a unique cluster.
6. The method of claim 1 , wherein within each cluster, a collection of similar images is maintained in a separate data structure.
7. The method of claim 1 , comprising mapping of facial feature vectors of received images to clusters in an off-line manner to improve accuracy and not affect foreground performance.
8. The method of claim 1 , during query time, comprising comparing a feature vector of the query face to all cluster centroids to find similar ones.
9. The method of claim 1 , wherein images belonging to cluster centroids are compared to the query face and images that match with a similarity score above a pre-defined threshold are deemed as potential matches and lookups are performed on-line to leverage an off-line constructed index and provide a real-time facial matching result.
10. The method of claim 1 , comprising maintaining a hierarchical index structure to improve a latency of real-time look ups.