IP Library Granted Patent US 9,916,494
Granted Patent B2
US 9,916,494 · App. 15/080,125 · Granted Mar 13, 2018

Positioning feature points of human face edge

Inventors: Jianfa Zou (Hangzhou, CN); Ying Huang (Hangzhou, CN); Yan Chen (Hangzhou, CN)
Assignee: Alibaba Group Holding Limited
G06K9/00281G06K9/6207G06T7/12G06T7/149G06K9/4642G06K9/4652G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,916,494
App. No.
15/080,125
Granted
Mar 13, 2018
Kind
B2
Abstract

An example method for positioning feature points of a human face edge including fitting a profile edge curve of a human face edge in a human face image according to the human face image; calculating by using a preset human face edge feature point calculation model to obtain feature points of the human face edge in the human face image; calculating by using a preset convergence algorithm to obtain feature information of a convergence feature point corresponding to each feature point; determining respectively whether a distance from each feature point to the profile edge curve is greater than a distance from a respective corresponding convergence feature point to the profile edge curve; and determining whether a magnitude of overall position change of all the feature points of the human face edge in the human face image before and after the above determination is less than a preset threshold.

Claims (72)

1. A method comprising:

fitting a profile edge curve of a human face edge in a human face image according to the human face image;

determining feature points of the human face edge in the human face image by using a preset human face edge feature point calculation model;

acquiring feature information of a respective feature point;

determining feature information of a convergence feature point corresponding to the respective feature point according to the feature information of the respective feature point;

determining whether a distance from the respective feature point to the profile edge curve is greater than a distance from a corresponding respective convergence feature point to the profile edge curve according to the feature information of the respective feature point and the respective convergence feature point;

acquiring the feature information of the respective convergence feature point corresponding to the feature point to serve as the feature information of the feature point, in response to determining that the distance from the respective feature point to the profile edge curve is greater than the distance from a corresponding respective convergence feature point to the profile edge curve; and

outputting the feature points and the feature information of the respective feature point through an output interface for determining a shape of the human face.

2. The method of claim 1 , further comprising keeping the feature information of the feature point unchanged, in response to determining that the distance from the respective feature point to the profile edge curve is not greater than the distance from a corresponding respective convergence feature point to the profile edge curve.

3. The method of claim 1 , further comprising:

determining whether a magnitude of overall position change of all the feature points of the human face edge in the human face image before and after the above determination is less than a preset threshold; and

using position information of the group of feature points as qualified feature points of human face edge, in response to determining that the magnitude is less than the preset threshold.

4. The method of claim 3 , further comprising:

using a preset convergence algorithm to obtain feature information of a convergence feature point corresponding to each feature point according to the feature information of each feature point, in response to determining that the magnitude is not less than the preset threshold.

5. The method of claim 3 , wherein the determining whether the magnitude of overall position change of all the feature points of the human face edge in the human face image before and after the above determination is less than the preset threshold includes:

calculating, according to feature information of all the feature points of the human face edge in the human face image before and after the above determination, the magnitude of overall position change of all the feature points of the human face edge in the human face image before and after the above determination;

determining whether the magnitude of the position change is less than the preset threshold; and

determining that feature information of each feature point of the human face edge in the human face image is unchanged, in response to determining that the magnitude is less than the preset threshold.

6. The method of claim 5 , further comprising returning to the calculating by using the preset convergence algorithm to obtain feature information of the convergence feature point corresponding to each feature point according to the feature information of each feature point, in response to determining that the magnitude is not less than the preset threshold.

7. The method of claim 5 , wherein the calculating the magnitude of overall position change of all the feature points of the human face edge in the human face image before and after the above determination includes:

acquiring a vector formed by connecting all feature points of the human face edge in the human face image before the above determination;

acquiring a vector formed by connecting all feature points of the human face edge in the human face image after the above determination; and

calculating a Euclidean distance between the vector formed by connecting all feature points of the human face edge in the human face image before the above determination and the vector formed by connecting all feature points of the human face edge in the human face image after the above determination to serve as the magnitude of overall position change of all the feature points of the human face edge in the human face image before and after the above determination.

8. The method of claim 1 , wherein the preset convergence algorithm includes:

Δ X 1 =R 0 *φ 0 +b 0 ; and

X k =X k−1 +R k−1 *φ k−1 +b k−1 ,

wherein:

X k is a vector formed by connecting convergence feature points obtained after calculation by using the convergence algorithm for the k th time, and X k−1 is a vector formed by connecting convergence feature points obtained after calculation by using the convergence algorithm for the (k−1) th time;

ΔX 1 is an offset value between a vector X 1 formed by connecting convergence feature points obtained after calculation by using the convergence algorithm for the first time and a vector X 0 formed by connecting the feature points of the human face edge in the human face image; φ 0 is a vector formed by connecting gradient direction features of the feature points before calculation by using the convergence algorithm for the first time; and R0 and b0 are preset coefficients; and

R k−1 and b k−1 are coefficients after calculation by using the convergence algorithm for the (k−1) th time, and φ k−1 is a vector formed by connecting gradient direction features of convergence feature points obtained after calculation by using the convergence algorithm for the (k−1) th time.

9. The method of claim 1 , further comprising:

prior to fitting the profile edge curve of human face edge in the human face image according to the human face image,

acquiring the human face image of the human face;

separating skin color regions and non-skin color regions in the human face image by using a preset skin color model; and

filtering out the non-skin color regions in the human face image.

10. The method of claim 9 , further comprising:

after acquiring the human face image of the human face,

expanding or clipping the acquired human face image into a human face image having a set size or format.

11. The method of claim 8 , wherein the preset skin color model includes a Gaussian mixture model.

12. The method of claim 1 , wherein the fitting the profile edge curve of human face edge in the human face image according to the human face image includes:

detecting strong edge of the human face in the human face image; and

fitting the profile edge curve of human face edge in the human face image according to the detected strong edge of the human face in the human face image.

13. The method of claim 12 , further comprising acquiring the strong edge of the human face in the human face image by detecting using a Canny edge detection operator.

14. The method of claim 12 , wherein the fitting the profile edge curve of human face edge in the human face image includes:

filtering the detected strong edge of the human face in the human face image to obtain the profile curve of the human face in the human face image;

making at least one ray according to a preset angle by using at least one feature point corresponding to the five sense organs of the human face in the human face image as an endpoint of the ray in the human face image corresponding to the profile curve of the human face; and

acquiring an intersection of the ray and the profile curve of the human face in the human face image and fitting a third-order Bezier curve in the human face image corresponding to the profile curve of the human face to serve as the profile edge curve of the human face edge in the human face image.

15. The method of claim 1 , wherein the calculating by using the preset human face edge feature point calculation model to obtain feature points of the human face edge in the human face image includes:

calculating by using the human face edge feature point calculation model in combination with pre-trained parameters to obtain position information of the respective feature point of the human face edge in the human face image;

acquiring a gradient direction feature of the respective feature point, acquired in the above calculating, in a region with the respective feature point itself as the center and having a set size or a set shape; and

using the position information and the gradient direction feature of the respective feature point as respective feature information of the respective feature point.

16. The method of claim 1 , wherein the feature information includes position information.

17. An apparatus comprising:

one or more processors; and

one or more memories stored thereon computer executable units executable by the one or more processors, the computer executable units including:

a profile edge curve fitting unit that detects a strong edge of a human face in the human face image; and fits a profile edge curve of a human face edge in the human face image according to the detected strong edge of the human face image;

a feature information calculation unit that calculates by using a preset human face edge feature point calculation model to obtain feature points of the human face edge in the human face image, and acquires feature information of a respective feature point, the feature information including at least position information;

a convergence calculation unit that calculates by using a preset convergence algorithm to obtain feature information of a convergence feature point corresponding to the respective feature point according to the feature information of the respective feature point;

a convergence direction determination unit that:

determines whether a distance from the respective feature point to the profile edge curve is greater than a distance from a corresponding respective convergence feature point to the profile edge curve according to the feature information of the respective feature point and the respective convergence feature point; and

acquires the feature information of the respective convergence feature point corresponding to the feature point to serve as the feature information of the feature point, in response to determining that the distance from the respective feature point to the profile edge curve is greater than the distance from a corresponding respective convergence feature point to the profile edge curve, and

an output interface that outputs the feature points and the feature information of the respective feature point for determining a shape of the human face.

18. The apparatus of claim 17 , wherein the computer executable units further comprise a feature information determination unit that uses position information of the group of feature points as qualified feature points of human face edge.

19. The apparatus of claim 17 , wherein the convergence direction determination unit keeps the feature information of the feature point unchanged, in response to determining that the distance from the respective feature point to the profile edge curve is not greater than the distance from a corresponding respective convergence feature point to the profile edge curve.

20. One or more memories stored thereon computer-executable instructions, executable by one or more processors, to cause the one or more processors to perform acts comprising:

fitting a profile edge curve of a human face edge in a human face image according to the human face image;

determining feature points of the human face edge in the human face image by using a preset human face edge feature point calculation model;

acquiring feature information of a respective feature point;

determining feature information of a convergence feature point corresponding to the respective feature point according to the feature information of the respective feature point;

determining whether a distance from the respective feature point to the profile edge curve is greater than a distance from a corresponding respective convergence feature point to the profile edge curve according to the feature information of the respective feature point and the respective convergence feature point;

acquiring the feature information of the respective convergence feature point corresponding to the feature point to serve as the feature information of the feature point, in response to determining that the distance from the respective feature point to the profile edge curve is greater than the distance from a corresponding respective convergence feature point to the profile edge curve; and

outputting the feature points and the feature information of the respective feature point through an output interface for determining a shape of the human face.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 25, 2020
From: ALIBABA GROUP HOLDING LIMITED
To: BANMA ZHIXING NETWORK (HONGKONG) CO., LIMITED
Reel/Frame 054469/0946 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 14, 2017
From: ZOU, JIANFA; HUANG, YING; CHEN, YAN
To: ALIBABA GROUP HOLDING LIMITED
Reel/Frame 043544/0228 →
Priority Claims (1)
CN 2015 1 0134068 · Mar 25, 2015 · national
Continuity (1)
Related Publication 20160283780A1 · Sep 29, 2016