IP Library Granted Patent US 11,651,229
Granted Patent B2
US 11,651,229 · App. 16/879,793 · Granted May 16, 2023

Methods and systems for face recognition

Inventors: Fuyun Cheng (Hangzhou, CN); Jingsong Hao (Hangzhou, CN)
Assignee: ZHEJIANG DAHUA TECHNOLOGY CO., LTD.
G06N3/084G06N3/045G06V10/454G06V10/462G06V10/50G06V10/763G06V10/764G06V10/806G06V10/82G06V40/168G06V40/172
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,651,229
App. No.
16/879,793
Granted
May 16, 2023
Kind
B2
Abstract

Systems and methods for face recognition are provided. The systems may perform the methods to obtain a neural network comprising a first sub-neural network and a second sub-neural network; generate a plurality of preliminary feature vectors based on an image associated with a human face, the plurality of preliminary feature vectors comprising a color-based feature vector; obtain at least one input feature vector based on the plurality of preliminary feature vectors; generate a deep feature vector based on the at least one input feature vector using the first sub-neural network; and recognize the human face based on the deep feature vector.

Claims (79)

1. A method, comprising:

obtaining a neural network comprising a first sub-neural network and a second sub-neural network, wherein the network layers contained in the first sub-neural network and the second sub-neural network are divided into a plurality of groups, at least one group of the plurality of groups corresponds to a region of interest, the region of interest is a sub-image of a target image, and the target image includes a standard image and a currently acquired image;

generating a plurality of preliminary feature vectors based on the sub-image, the plurality of preliminary feature vectors comprising a color-based feature vector, and the plurality of preliminary feature vectors including standard preliminary feature vectors and currently acquired image preliminary feature vectors;

obtaining at least one input feature vector based on the plurality of preliminary feature vectors;

generating a plurality of deep feature vectors based on the at least one input feature vector using the network layers contained in the first sub-neural network in the at least one group, the plurality of deep feature vectors including standard deep feature vectors and currently acquired image deep feature vectors; and

recognizing a human face based on the plurality of deep feature vectors by:

generating a plurality of matching scores based on the standard deep feature vectors and the currently acquired image deep feature vectors using the network layers contained in the second sub-neural network in the at least one group; combining the plurality of matching scores to generate a final matching score; and

recognizing the human face based on the final matching score.

2. The method of claim 1 , wherein the recognizing a human face based on the plurality of deep feature vectors comprises:

generating an output using the network layers contained in the second sub-neural network in the at least one group based on the plurality of deep feature vectors; and

recognizing the human face based on the output.

3. The method of claim 2 , wherein the recognizing a human face based on the plurality of deep feature vectors further comprises:

determining a pose of the human face based on the output.

4. The method of claim 3 , wherein the output comprises at least one posing parameter, and wherein the posing parameter comprises at least one of a yaw parameter or a pitch parameter.

5. The method of claim 2 , wherein the generating an output using the second sub-neural network based on the plurality of deep feature vectors comprises:

fusing the plurality of deep feature vectors to form an ultimate feature vector; and

generating the output using the second sub-neural network based on the ultimate feature vector.

6. The method of claim 2 , wherein the target image includes a first image and a second image, and the generating a plurality of preliminary feature vectors includes further:

generating a plurality of first sub-images based on the first image;

generating a plurality of first preliminary feature vectors based on at least one of the plurality of the first sub-images;

generating a plurality of second sub-images based on the second image;

generating a plurality of second preliminary feature vectors based on at least one of the plurality of second sub-images; and

the obtaining at least one input feature vector based on the plurality of preliminary feature vectors includes:

obtaining at least one first input feature vector based on the plurality of first preliminary feature vectors;

obtaining at least one second input feature vector based on the plurality of the second preliminary feature vectors;

the generating a plurality of deep feature vectors based on the at least one input feature vector includes:

generating a first deep feature vector based on the at least one first input feature vector using the network layers contained in the first sub-neural network in the at least one group;

generating a second deep feature vector based on the at least one second input feature vector through the network layers contained in the first sub-neural network in the at least one group; and

the generating an output using the network layers contained in the second sub-neural network in the at least one group based on the one or more deep feature vectors includes:

generating the output using the network layers contained in the second sub-neural network in the at least one group based on the first deep feature vector and the second deep feature vector.

7. The method of claim 6 , wherein the generating the output using the network layers contained in the second sub-neural network in the at least one group based on the first deep feature vector and the second deep feature vector further comprises:

generating a first intermediate associated with at least one of the plurality of second sub-images based on the first deep feature vector and the second deep feature vector;

generating a second intermediate based on the first intermediates associated with the at least one of the second sub-images; and

generating the output based on the second intermediate.

8. The method of claim 1 , wherein the first sub-neural network includes one or more secondary sub-neural networks with convolutional network architecture, and wherein the one or more secondary sub-neural networks include a feature layer configured to generate the one or more deep feature vectors.

9. The method of claim 8 , wherein the feature layer is fully connected to a layer within at least one of the one or more secondary sub-neural networks.

10. The method of claim 9 , further comprising:

training the neural network by performing a backpropagation operation, comprising:

determining an error at the feature layer of the one or more secondary sub-neural networks;

dividing the error into a plurality of error portions, wherein the number of the error portions corresponds to the number of the one or more secondary sub-neural networks; and

performing the backpropagation operation on the one or more secondary sub-neural networks based on the plurality of error portions.

11. The method of claim 10 , further comprising:

dividing the error into the plurality of error portions based on the number of neural units of the feature layer of the one or more secondary sub-neural networks.

12. The method of claim 1 , wherein the obtaining at least one input feature vector based on the plurality of preliminary feature vectors comprises:

using at least one of the plurality of preliminary feature vectors as the at least one input feature vector.

13. The method of claim 12 , wherein the plurality of preliminary feature vectors includes at least one of a texture-based feature vector or a gradient-based feature vector.

14. The method of claim 1 , wherein the obtaining at least one input feature vector based on the plurality of preliminary feature vectors further comprises:

generating a combined preliminary feature vector by stacking at least two of the plurality of preliminary feature vectors; and

using the combined preliminary feature vector as the at least one input feature vector.

15. The method of claim 14 , wherein the plurality of preliminary feature vectors includes at least one of a first texture-based feature vector or a second texture-based feature vector.

16. The method of claim 1 , wherein the generating a plurality of preliminary feature vectors includes:

generating a plurality of sub-images based on the target image, wherein the plurality of sub-images corresponds to a plurality of parts of the target image; and

generating the plurality of preliminary feature vectors based on at least one of the plurality of the sub-images.

17. The method of claim 1 , further comprising:

training at least part of the neural network comprising the first sub-neural network and the second sub-neural network; and

tuning the at least part of the neural network.

18. The method of claim 17 , wherein the tuning the at least part of the neural network further comprises:

obtaining a plurality of second features at a first feature layer of the first sub-neural network or a layer connecting to the feature layer;

obtaining a plurality of normalized features by normalizing the plurality of second features;

clustering the normalized features into at least one cluster, the cluster comprising a feature determined as a centroid; and

tuning the at least part of the neural network based on at least one centroid.

19. A system, comprising:

at least one non-transitory computer-readable storage medium configured to store data and instructions; and

at least one processor in communication with the at least one non-transitory computer-readable storage medium, wherein when executing the instructions, the at least one processor is directed to:

obtaining a neural network comprising a first sub-neural network and a second sub-neural network, wherein the network layers contained in the first sub-neural network and the second sub-neural network are divided into a plurality of groups, at least one group of the plurality of groups corresponds to a region of interest, the region of interest is a sub-image of a target image, and the target image includes a standard image and a currently acquired image;

generating a plurality of preliminary feature vectors based on the sub-image, the plurality of preliminary feature vectors comprising a color-based feature vector, and the plurality of preliminary feature vectors including standard preliminary feature vectors and currently acquired image preliminary feature vectors;

obtaining at least one input feature vector based on the plurality of preliminary feature vectors;

generating a plurality of deep feature vectors based on the at least one input feature vector using the network layers contained in the first sub-neural network in the at least one group, the plurality of deep feature vectors including standard deep feature vectors and currently acquired image deep feature vectors; and

recognizing a human face based on the plurality of deep feature vectors by:

generating a plurality of matching scores based on the standard deep feature vectors and the currently acquired image deep feature vectors using the network layers contained in the second sub-neural network in the at least one group; combining the plurality of matching scores to generate a final matching score; and

recognizing the human face based on the final matching score.

20. A non-transitory computer readable medium, comprising executable instructions that, when executed by at least one processor, direct the at least one processor to perform a method, the method comprising:

obtaining a neural network comprising a first sub-neural network and a second sub-neural network, wherein the network layers contained in the first sub-neural network and the second sub-neural network are divided into a plurality of groups, at least one group of the plurality of groups corresponds to a region of interest, the region of interest is a sub-image of a target image, and the target image includes a standard image and a currently acquired image;

generating a plurality of preliminary feature vectors based on the sub-image, the plurality of preliminary feature vectors comprising a color-based feature vector, and the plurality of preliminary feature vectors including standard preliminary feature vectors and currently acquired image preliminary feature vectors;

obtaining at least one input feature vector based on the plurality of preliminary feature vectors;

generating a plurality of deep feature vectors based on the at least one input feature vector using the network layers contained in the first sub-neural network in the at least one group, the plurality of deep feature vectors including standard deep feature vectors and currently acquired image deep feature vectors; and

recognizing a human face based on the plurality of deep feature vectors by:

generating a plurality of matching scores based on the standard deep feature vectors and the currently acquired image deep feature vectors using the network layers contained in the second sub-neural network in the at least one group; combining the plurality of matching scores to generate a final matching score; and

recognizing the human face based on the final matching score.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2020
From: CHENG, FUYUN; HAO, JINGSONG
To: ZHEJIANG DAHUA TECHNOLOGY CO., LTD.
Reel/Frame 052742/0068 →
Priority Claims (3)
CN 201711174440.9 · Nov 22, 2017 · national
CN 201711174490.7 · Nov 22, 2017 · national
CN 201711176849.4 · Nov 22, 2017 · national
Continuity (2)
Continuation PCTCN2017114140 · Nov 30, 2017
Related Publication 20200327308A1 · Oct 15, 2020
Cited By (1)
US 12,725,271