IP Library Granted Patent US 12,148,250
Granted Patent B2
US 12,148,250 · App. 17/685,177 · Granted Nov 19, 2024

AI-based face recognition method and apparatus, device, and medium

Inventors: Yuan Gao (Shenzhen, CN); Zhifeng Li (Shenzhen, CN); Wei Liu (Shenzhen, CN)
Assignee: Tencent Technology (Shenzhen) Company Limited
G06V40/45G06V10/56G06V10/774G06V10/776G06V10/82G06V20/64G06V40/171G06V40/172G06V40/20G06F21/32
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,148,250
App. No.
17/685,177
Granted
Nov 19, 2024
Kind
B2
Abstract

An artificial intelligence (AI)-based face recognition method includes: obtaining n groups of input video frames, at least one group of video frames including a color video frame and a depth video frame of a target face, n being a positive integer; invoking an interactive liveness detection function to recognize the color video frames in the n groups of video frames; invoking a second three-dimensional (3D) structured-light liveness detection function to recognize the depth video frames in the n groups of video frames; and determining, in response to both detection results of the interactive liveness detection function and the 3D structured-light liveness detection function indicating that a type of the target face being a liveness type, that the target face is a live target face.

Claims (82)

1. An artificial intelligence (AI)-based face recognition method, applicable to a computer device, the method comprising:

obtaining n groups of input video frames, at least one group of video frames comprising a color video frame and a depth video frame of a target face, n being a positive integer;

invoking an interactive liveness detection function to recognize the color video frames in the n groups of video frames, comprising:

invoking a neural network model in the interactive liveness detection function to obtain a position of a facial feature point on each of the color video frames; and

invoking the interactive liveness detection function to determine, according to a distance between the facial feature points of the color video frames, whether the target face completes a target action;

invoking a second three-dimensional (3D) structured-light liveness detection function to recognize the depth video frames in the n groups of video frames; and

determining, in response to both detection results of the interactive liveness detection function and the 3D structured-light liveness detection function indicating that a type of the target face being a liveness type, that the target face is a live target face,

wherein the neural network model is obtained by:

obtaining a training sample set, the training sample set comprising a plurality of groups of sample face images and sample facial feature points;

invoking the interactive liveness detection function to recognize a sample face image of the plurality of groups of sample face images, to obtain a predicted facial feature point of a sample face;

comparing a sample facial feature point, of the sample facial feature points, corresponding to the sample face with the predicted facial feature point of the sample face to calculate an error loss; and

training the neural network model of the interactive liveness detection function based on the error loss and by using an error backpropagation algorithm to obtain a trained neural network model of the interactive liveness detection function.

2. The method according to claim 1 , wherein the invoking the interactive liveness detection function to determine, according to a distance between the facial feature points on the color video frames, whether the target face completes a target action comprises:

invoking the interactive liveness detection function to calculate a ratio of a maximum horizontal distance to a maximum longitudinal distance between the facial feature points belonging to a same facial feature part, the same facial feature part comprising at least one of an eye or a mouth; and

determining, in response to the ratio reaching a preset condition, that the target face completes the target action.

3. The method according to claim 2 , wherein the determining, in response to the ratio reaching a preset condition, that the target face completes the target action comprises:

determining, in response to the facial feature part being the eye and the ratio reaching a first distance threshold, that the target face completes a blinking action; and

determining, in response to the facial feature part being the mouth and the ratio reaching a second distance threshold, that the target face completes a mouth opening action.

4. The method according to claim 3 , wherein the facial feature points comprise an eye feature point, and the eye feature point comprises a left canthus feature point, a right canthus feature point, an upper eyelid feature point, and a lower eyelid feature point; and

the determining, in response to the facial feature part being the eye and the ratio reaching a first distance threshold, that the target face completes a blinking action comprises:

obtaining a first distance and a second distance, the first distance being a distance between the left canthus feature point and the right canthus feature point, and the second distance being a distance between the upper eyelid feature point and the lower eyelid feature point;

calculating a distance ratio between the first distance and the second distance;

calculating a mean value of the distance ratios of two eyes on the target face; and

determining, in response to the mean value being greater than the first distance threshold, that the target face completes the blinking action.

5. The method according to claim 2 , wherein the interactive liveness detection function further comprises: at least one of a first classifier or a second classifier, and the determining, in response to the ratio reaching a preset condition, that the target face completes the target action comprises:

determining, in response to the facial feature part being the eye and the ratio being recognized by the first classifier as a first type, that the target face completes a blinking action; and

determining, in response to the facial feature part being the mouth and the ratio being recognized by the second classifier as a second type, that the target face completes a mouth opening action.

6. The method according to claim 1 , further comprising:

invoking a face preprocessing function to recognize the color video frames in the n groups of video frames, the face preprocessing function being a machine learning function with a face recognition capability;

obtaining a recognition result that there are m groups of video frames having a face region, m≤n, and m being a positive integer; and

providing color video frames in the m groups of video frames to the interactive liveness detection function, and providing depth video frames in the m groups of video frames to the 3D structured-light liveness detection function.

7. The method according to claim 6 , further comprising:

in response to m being less than a third preset threshold, filtering out the n groups of video frames.

8. The method according to claim 1 , wherein the determining, in response to both detection results of the interactive liveness detection function and the 3D structured-light liveness detection function indicating that a type of the target face being a liveness type, that the target face is a live target face comprises:

obtaining a first frame quantity and a second frame quantity, the first frame quantity being a quantity of color video frames comprising the target action and recognized by the interactive liveness detection function, and the second frame quantity being a quantity of depth video frames recognized by the 3D structured-light liveness detection function and conforming to depth information corresponding to the live face; and

determining, in response to the first frame quantity being greater than a first preset threshold and the second frame quantity being greater than a second preset threshold, that the target face is the live target face.

9. The method according to claim 8 , further comprising:

determining, in response to the first frame quantity of the video frames being less than the first preset threshold, that the target face is a non-live target face, and filtering out the n groups of video frames;

or,

determining, in response to the second frame quantity of the video frames being less than the second preset threshold, that the target face is a non-live target face, and filtering out the n groups of video frames.

10. An artificial intelligence (AI)-based face recognition apparatus, comprising a processor and a memory, the memory storing one or more computer programs, the processor being configured to load and execute the one or more computer programs to implement:

obtaining n groups of input video frames, at least one group of video frames comprising a color video frame and a depth video frame of a target face, n being a positive integer;

invoking an interactive liveness detection function to recognize the color video frames in the n groups of video frames, comprising:

invoking a neural network model in the interactive liveness detection function to obtain a position of a facial feature point on each of the color video frames; and

invoking the interactive liveness detection function to determine, according to a distance between the facial feature points of the color video frames, whether the target face completes a target action;

invoking a second three-dimensional (3D) structured-light liveness detection function to recognize the depth video frames in the n groups of video frames; and

determining, in response to both detection results of the interactive liveness detection function and the 3D structured-light liveness detection function indicating that a type of the target face being a liveness type, that the target face is a live target face,

wherein the neural network model is obtained by:

obtaining a training sample set, the training sample set comprising a plurality of groups of sample face images and sample facial feature points;

invoking the interactive liveness detection function to recognize a sample face image of the plurality of groups of sample face images, to obtain a predicted facial feature point of a sample face;

comparing a sample facial feature point, of the sample facial feature points, corresponding to the sample face with the predicted facial feature point of the sample face to calculate an error loss; and

training the neural network model of the interactive liveness detection function based on the error loss and by using an error backpropagation algorithm to obtain a trained neural network model of the interactive liveness detection function.

11. The apparatus according to claim 10 , wherein the invoking the interactive liveness detection function to determine, according to a distance between the facial feature points on the color video frames, whether the target face completes a target action comprises:

invoking the interactive liveness detection function to calculate a ratio of a maximum horizontal distance to a maximum longitudinal distance between the facial feature points belonging to a same facial feature part, the same facial feature part comprising at least one of an eye or a mouth; and

determining, in response to the ratio reaching a preset condition, that the target face completes the target action.

12. The apparatus according to claim 11 , wherein the determining, in response to the ratio reaching a preset condition, that the target face completes the target action comprises:

determining, in response to the facial feature part being the eye and the ratio reaching a first distance threshold, that the target face completes a blinking action; and

determining, in response to the facial feature part being the mouth and the ratio reaching a second distance threshold, that the target face completes a mouth opening action.

13. The apparatus according to claim 12 , wherein the facial feature points comprise an eye feature point, and the eye feature point comprises a left canthus feature point, a right canthus feature point, an upper eyelid feature point, and a lower eyelid feature point; and

the determining, in response to the facial feature part being the eye and the ratio reaching a first distance threshold, that the target face completes a blinking action comprises:

obtaining a first distance and a second distance, the first distance being a distance between the left canthus feature point and the right canthus feature point, and the second distance being a distance between the upper eyelid feature point and the lower eyelid feature point;

calculating a distance ratio between the first distance and the second distance;

calculating a mean value of the distance ratios of two eyes on the target face; and

determining, in response to the mean value being greater than the first distance threshold, that the target face completes the blinking action.

14. The apparatus according to claim 11 , wherein the interactive liveness detection function further comprises: at least one of a first classifier or a second classifier, and the determining, in response to the ratio reaching a preset condition, that the target face completes the target action comprises:

determining, in response to the facial feature part being the eye and the ratio being recognized by the first classifier as a first type, that the target face completes a blinking action; and

determining, in response to the facial feature part being the mouth and the ratio being recognized by the second classifier as a second type, that the target face completes a mouth opening action.

15. The apparatus according to claim 10 , wherein the determining, in response to both detection results of the interactive liveness detection function and the 3D structured-light liveness detection function indicating that a type of the target face being a liveness type, that the target face is a live target face comprises:

obtaining a first frame quantity and a second frame quantity, the first frame quantity being a quantity of color video frames comprising the target action and recognized by the interactive liveness detection function, and the second frame quantity being a quantity of depth video frames recognized by the 3D structured-light liveness detection function and conforming to depth information corresponding to the live face; and

determining, in response to the first frame quantity being greater than a first preset threshold and the second frame quantity being greater than a second preset threshold, that the target face is the live target face.

16. A non-transitory computer-readable storage medium, storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by a processor to implement:

obtaining n groups of input video frames, at least one group of video frames comprising a color video frame and a depth video frame of a target face, n being a positive integer;

invoking an interactive liveness detection function to recognize the color video frames in the n groups of video frames, comprising:

invoking a neural network model in the interactive liveness detection function to obtain a position of a facial feature point on each of the color video frames; and

invoking the interactive liveness detection function to determine, according to a distance between the facial feature points of the color video frames, whether the target face completes a target action;

invoking a second three-dimensional (3D) structured-light liveness detection function to recognize the depth video frames in the n groups of video frames; and

determining, in response to both detection results of the interactive liveness detection function and the 3D structured-light liveness detection function indicating that a type of the target face being a liveness type, that the target face is a live target face,

wherein the neural network model is obtained by:

obtaining a training sample set, the training sample set comprising a plurality of groups of sample face images and sample facial feature points;

invoking the interactive liveness detection function to recognize a sample face image of the plurality of groups of sample face images, to obtain a predicted facial feature point of a sample face;

comparing a sample facial feature point, of the sample facial feature points, corresponding to the sample face with the predicted facial feature point of the sample face to calculate an error loss; and

training the neural network model of the interactive liveness detection function based on the error loss and by using an error backpropagation algorithm to obtain a trained neural network model of the interactive liveness detection function.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2022
From: GAO, YUAN; LI, ZHIFENG; LIU, WEI
To: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED
Reel/Frame 059151/0786 →
Priority Claims (1)
CN 202010075684.7 · Jan 22, 2020 · national
Continuity (2)
Continuation PCTCN2020124944 · Oct 29, 2020
Related Publication 20220309836A1 · Sep 29, 2022
Cited By (1)
US 12,646,256