Facial verification method and apparatus based on three-dimensional (3D) image

1. A facial verification method comprising:

capturing a facial image including a three-dimensional (3D) face of a user;

determining an occluded region in the captured facial image by comparing the captured facial image and an average facial image;

generating a synthetic image by synthesizing the captured facial image and the average facial image based on the determined occluded region;

determining a reliability of the captured facial image based on at least one of a ratio of the occluded region in the captured facial image and a position of the occluded region in the captured facial image; and

verifying the user based on the determined reliability of the captured facial image using the generated synthetic image,

wherein the verifying comprising:

extracting features from the captured facial image to perform the verifying in response to the reliability of the captured facial image being greater than a threshold value; and

terminating the verifying using the captured facial image without extracting features from the captured facial image in response to the reliability of the captured facial image being less than the threshold value.

2. The facial verification method of claim 1 , wherein the determining of the occluded region comprises:

determining the occluded region by comparing pixel values of corresponding pixels in the captured facial image and the average facial image.

3. The facial verification method of claim 1 , wherein the generating of the synthetic image comprises:

determining a replacement region in the average facial image corresponding to the occluded region; and

generating the synthetic image by replacing the occluded region in the captured facial image with the determined replacement region.

4. The facial verification method of claim 1 , further comprising:

determining at least one of the ratio of the occluded region in the captured facial image and the position of the occluded region in the captured facial image.

5. The facial verification method of claim 4 , wherein the ratio of the occluded region is determined based on at least one of a ratio of a size of the occluded region to a size of the captured facial image, and a ratio of a number of landmarks included in the occluded region to a number of landmarks included in the captured facial image.

6. The facial verification method of claim 4 , wherein the position of the occluded region is determined based on positions of landmarks included in the captured facial image.

7. The facial verification method of claim 4 , wherein the verifying of the user comprises:

determining the reliability of the captured facial image based on at least one of the ratio of the occluded region and the position of the occluded region;

calculating a score to be used to verify the user; and

verifying the user by applying, to the score, a weighted value corresponding to the reliability of the captured facial image.

8. The facial verification method of claim 1 , wherein the determined reliability of the captured facial image decreases as the ratio of the occluded region increases and the occluded region is closer to a major facial landmark.

9. The facial verification method of claim 1 , wherein the average facial image is generated based on an average pixel value of corresponding pixels in 3D faces included in reference images.

10. The facial verification method of claim 1 , wherein the occluded region includes at least one of a region corresponding to an occluding element covering the 3D face of the user in the captured facial image, a region corresponding to a portion of an actual face of the user that is not included in the captured facial image, or a region corresponding to landmarks deviating far from a reference arrangement of landmarks in the captured facial image.

11. The facial verification method of claim 1 , wherein the obtaining of the captured facial image comprises:

receiving an input image including the 3D face of the user;

detecting the 3D face of the user from the input image;

detecting facial landmarks from the detected 3D face of the user;

separating the 3D face of the user from the input image based on the detected facial landmarks; and

generating the captured facial image by performing normalization on the separated 3D face of the user based on the detected landmarks.

12. The facial verification method of claim 1 , wherein the verifying of the user comprises:

verifying the user by comparing features extracted from the synthetic image and features extracted from a registered image of the user.

13. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the facial verification method of claim 1 .

14. A facial verification apparatus comprising:

a processor configured to:

capture a facial image including a three-dimensional (3D) face of a user;

determine an occluded region in the captured facial image by comparing the captured facial image and an average facial image;

generate a synthetic image by synthesizing the captured facial image and the average facial image based on the determined occluded region;

determining a reliability of the captured facial image based on at least one of a ratio of the occluded region in the captured facial image and a position of the occluded region in the captured facial image; and

verify the user based on the determined reliability of the captured facial image using the generated synthetic image

wherein the processor is further configured to:

extract features from the captured facial image to perform the verifying in response to the reliability of the captured facial image being greater than a threshold value; and

terminate the verifying using the captured facial image without extracting features from the captured facial image in response to the reliability of the captured facial image being less than the threshold value.

15. The facial verification apparatus of claim 14 , wherein the processor is further configured to determine at least one of the ratio of the occluded region in the captured facial image and the position of the occluded region in the captured facial image.

16. The facial verification apparatus of claim 15 , wherein the processor is further configured to determine the reliability of the captured facial image based on at least one of the ratio of the occluded region and the position of the occluded region, calculate a score to be used to verify the user, and verify the user by applying, to the score, a weighted value corresponding to the reliability of the captured facial image.