IP Library › Granted Patent US 11,734,954
Granted Patent B2
US 11,734,954 · App. 17/266,587 · Granted Aug 22, 2023

Face recognition method, device and electronic equipment, and computer non-volatile readable storage medium

Inventors: Moyan Zhao (Guangdong, CN); Hongwei Wang (Guangdong, CN)
Assignee: PING AN TECHNOLOGY (SHENZHEN) CO., LTD.
G06V40/171G06V10/751G06V40/172
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,734,954
App. No.
17/266,587
Granted
Aug 22, 2023
Kind
B2
Abstract

A face recognition method includes: detecting keypoints when receiving a first face image; acquiring a recognition score of each detectable keypoint and serial numbers of missing keypoints; acquiring a plurality of target keypoints in the plurality of detectable keypoints having a predetermined face feature association relationship with the missing keypoints when the influence score is higher than a predetermined score threshold; acquiring a target face feature template having a degree of position combination with the plurality of target keypoints greater than a predetermined combination degree threshold; and stitching the target face feature template and the plurality of target keypoints on the first face image to obtain a second face image so as to detect all the keypoints according to the second face image for performing the face recognition.

Claims (59)

1. A face recognition method, comprising:

detecting all keypoints from a first face image to obtain a plurality of detectable keypoints and coordinates of each detectable keypoint when receiving the first face image with blocked partial face features;

acquiring a recognition score of each detectable keypoint and serial numbers of missing keypoints from a preset keypoint information database based on each detectable keypoint;

acquiring an influence score of an area with the blocked partial face features on the face recognition according to the recognition score of each detectable keypoint;

acquiring a plurality of target keypoints in the plurality of detectable keypoints having a predetermined face feature association relationship with the missing keypoints based on the serial numbers of the missing keypoints when the influence score is higher than a predetermined score threshold;

acquiring a target face feature template having a degree of position combination with the plurality of target keypoints greater than a predetermined combination degree threshold from a predetermined face feature template library according to the coordinates of the plurality of target keypoints; and

stitching the target face feature template and the plurality of target keypoints on the first face image to obtain a second face image so as to detect all the keypoints according to the second face image for performing the face recognition.

2. The method according to claim 1 , wherein the acquiring a recognition score of each detectable keypoint and serial numbers of missing keypoints from a preset keypoint information database based on each detectable keypoint comprises:

sequentially searching each detectable keypoint from a preset keypoint information database to acquire a recognition score stored in association with each detectable keypoint; and

taking other keypoints except the detectable keypoints in the preset keypoint information database as the missing keypoints.

3. The method according to claim 1 , wherein the acquiring an influence score of an area with the blocked partial face features on the face recognition according to the recognition score of each detectable keypoint comprises:

summing the recognition scores of all the detectable keypoints to obtain a recognition score sum; and

subtracting a predetermined recognition score threshold from the recognition score sum to obtain a difference value between the recognition score sum and the predetermined recognition score threshold, and taking the difference value as an influence score of an area with the blocked partial face features on the face recognition.

4. The method according to claim 1 , wherein the acquiring a plurality of target keypoints in the plurality of detectable keypoints having a predetermined face feature association relationship with the missing keypoints based on the serial numbers of the missing keypoints when the influence score is higher than a predetermined score threshold comprises:

dividing the missing keypoints derived from the same face feature into a group when the influence score is higher than a predetermined score threshold;

searching the serial numbers of all the keypoints of the same group derived from the same face feature as the serial numbers of the missing keypoints of each group from the preset keypoint information database; and

taking other keypoints with the serial numbers of the missing keypoints in each group removed from the serial numbers of all the keypoints of the same group as the target keypoints corresponding to the missing keypoints in each group.

5. The method according to claim 1 , wherein the acquiring a plurality of target keypoints in the plurality of detectable keypoints having a predetermined face feature association relationship with the missing keypoints based on the serial numbers of the missing keypoints when the influence score is higher than a predetermined score threshold comprises:

searching the serial numbers of all the keypoints derived from the same face feature as the serial number of each missing keypoint from the preset keypoint information database based on the serial numbers of the missing keypoints when the influence score is higher than a predetermined score threshold; and

acquiring a plurality of target keypoints from the plurality of detectable keypoints having a predetermined face feature association relationship with each missing keypoint according to the serial numbers of all the keypoints derived from the same face feature as the serial number of each missing keypoint.

6. The method according to claim 5 , wherein the acquiring a plurality of target keypoints from the plurality of detectable keypoints having a predetermined face feature association relationship with each missing keypoint according to the serial numbers of all the keypoints derived from the same face feature as the serial number of each missing keypoint comprises:

searching the serial numbers of all the keypoints derived from the same face feature as the serial number of each missing keypoint from the plurality of detectable keypoints to obtain the serial numbers of the searched detectable keypoints; and

taking the detectable keypoints corresponding to the serial numbers of the searched detectable keypoints as a plurality of target keypoints having a predetermined face feature association relationship with each missing keypoint.

7. The method according to claim 1 , wherein the acquiring a target face feature template having a degree of position combination with the plurality of target keypoints greater than a predetermined combination degree threshold from a predetermined face feature template library according to the coordinates of the plurality of target keypoints comprises:

acquiring the coordinates of the plurality of target keypoints derived from the target human face feature according to the coordinates of each target keypoint;

calculating a first distance between each two keypoints in the plurality of target keypoints according to the coordinates of the plurality of target keypoints derived from the target face feature;

acquiring a second distance between every two keypoints of the plurality of preset keypoints having the same serial numbers as the plurality of target keypoints in a predetermined face feature template set corresponding to the target face feature from a predetermined face feature template library;

calculating a degree of position combination of the plurality of target keypoints with each template in the predetermined face feature template set according to the first distance and the second distance; and

taking a template having a degree of position combination with the plurality of target keypoints greater than a predetermined combination degree threshold from the predetermined face feature template set as the target face feature template.

8. An electronic equipment, comprising a processing unit; and a storage unit configured for storing a face recognition program of the processing unit; wherein the processing unit is configured for performing the following processing via execution of the face recognition program:

detecting all keypoints from a first face image to obtain a plurality of detectable keypoints and coordinates of each detectable keypoint when receiving the first face image with blocked partial face features;

acquiring a recognition score of each detectable keypoint and serial numbers of missing keypoints from a preset keypoint information database based on each detectable keypoint;

acquiring an influence score of an area with the blocked partial face features on the face recognition according to the recognition score of each detectable keypoint;

acquiring a plurality of target keypoints in the plurality of detectable keypoints having a predetermined face feature association relationship with the missing keypoints based on the serial numbers of the missing keypoints when the influence score is higher than a predetermined score threshold;

acquiring a target face feature template having a degree of position combination with the plurality of target keypoints greater than a predetermined combination degree threshold from a predetermined face feature template library according to the coordinates of the plurality of target keypoints; and

stitching the target face feature template and the plurality of target keypoints on the first face image to obtain a second face image so as to detect all the keypoints according to the second face image for performing the face recognition.

9. The electronic equipment according to claim 8 , wherein the acquiring a recognition score of each detectable keypoint and serial numbers of missing keypoints from a preset keypoint information database based on each detectable keypoint comprises:

sequentially searching each detectable keypoint from a preset keypoint information database to acquire a recognition score stored in association with each detectable keypoint; and

taking other keypoints except the detectable keypoints in the preset keypoint information database as the missing keypoints.

10. The electronic equipment according to claim 8 , wherein the acquiring an influence score of an area with the blocked partial face features on the face recognition according to the recognition score of each detectable keypoint comprises:

summing the recognition scores of all the detectable keypoints to obtain a recognition score sum; and

subtracting a predetermined recognition score threshold from the recognition score sum to obtain a difference value between the recognition score sum and the predetermined recognition score threshold, and taking the difference value as an influence score of an area with the blocked partial face features on the recognized face.

11. The electronic equipment according to claim 8 , wherein the acquiring a plurality of target keypoints in the plurality of detectable keypoints having a predetermined face feature association relationship with the missing keypoints based on the serial numbers of the missing keypoints when the influence score is higher than a predetermined score threshold comprises:

dividing the missing keypoints derived from the same face feature into a group when the influence score is higher than a predetermined score threshold;

searching the serial numbers of all the keypoints of the same group derived from the same face feature as the serial numbers of the missing keypoints of each group from the preset keypoint information database; and

taking other keypoints with the serial numbers of the missing keypoints in each group removed from the serial numbers of all the keypoints of the same group as the target keypoints corresponding to the missing keypoints in each group.

12. The electronic equipment according to claim 8 , wherein the acquiring a plurality of target keypoints in the plurality of detectable keypoints having a predetermined face feature association relationship with the missing keypoints based on the serial numbers of the missing keypoints when the influence score is higher than a predetermined score threshold comprises:

searching the serial numbers of all the keypoints derived from the same face feature as the serial number of each missing keypoint from the preset keypoint information database based on the serial numbers of the missing keypoints when the influence score is higher than a predetermined score threshold; and

acquiring a plurality of target keypoints from the plurality of detectable keypoints having a predetermined face feature association relationship with each missing keypoint according to the serial numbers of all the keypoints derived from the same face feature as the serial number of each missing keypoint.

13. The electronic equipment according to claim 12 , wherein the acquiring a plurality of target keypoints from the plurality of detectable keypoints having a predetermined face feature association relationship with each missing keypoint according to the serial numbers of all the keypoints derived from the same face feature as the serial number of each missing keypoint comprises:

searching the serial numbers of all the keypoints derived from the same face feature as the serial number of each missing keypoint from the plurality of detectable keypoints to obtain the serial numbers of the searched detectable keypoints; and

taking the detectable keypoints corresponding to the serial numbers of the searched detectable keypoints as a plurality of target keypoints having a predetermined face feature association relationship with each missing keypoint.

14. The electronic equipment according to claim 8 , wherein the acquiring a target face feature template having a degree of position combination with the plurality of target keypoints greater than a predetermined combination degree threshold from a predetermined face feature template library according to the coordinates of the plurality of target keypoints comprises:

acquiring the coordinates of the plurality of target keypoints derived from the target human face feature according to the coordinates of each target keypoint;

calculating a first distance between each two keypoints in the plurality of target keypoints according to the coordinates of the plurality of target keypoints derived from the target face feature;

acquiring a second distance between every two keypoints of the plurality of preset keypoints having the same serial numbers as the plurality of target keypoints in a predetermined face feature template set corresponding to the target face feature from a predetermined face feature template library;

calculating a degree of position combination of the plurality of target keypoints with each template in the predetermined face feature template set according to the first distance and the second distance; and

taking a template having a degree of position combination with the plurality of target keypoints greater than a predetermined combination degree threshold from the predetermined face feature template set as the target face feature template.

15. A non-transitory computer readable storage medium having stored thereon a face recognition program which, when executed by a processing unit, performs the method according to claim 1 .

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 26, 2023
From: WANG, HONGWEI
To: PING AN TECHNOLOGY (SHENZHEN) CO., LTD.
Reel/Frame 064064/0871 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2021
From: ZHAO, MOYAN
To: PING AN TECHNOLOGY (SHENZHEN) CO., LTD.
Reel/Frame 055174/0162 →
Priority Claims (1)
CN 201910709169.7 · Aug 1, 2019 · national
Continuity (1)
Related Publication 20210312163A1 · Oct 7, 2021
Cited By (1)
US 12,356,064